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To 
Jane and Stephen 


PREFACE 


This book is a continuation of the author’s Calculus, Volume I, Second Edition. The 
present volume has been written with the same underlying philosophy that prevailed in the 
first. Sound training in technique is combined with a strong theoretical development. 
Every effort has been made to convey the spirit of modern mathematics without undue 
emphasis on formalization. As in Volume I, historical remarks are included to give the 
student a sense of participation in the evolution of ideas. 

The second volume is divided into three parts, entitled Linear Analysis, Nonlinear 
Analysis, and Special Topics. The last two chapters of Volume I have been repeated as the 
first two chapters of Volume II so that all the material on linear algebra will be complete 
in one volume. 

Part 1 contains an introduction to linear algebra, including linear transformations, 
matrices, determinants, eigenvalues, and quadratic forms. Applications are given to 
analysis, in particular to the study of linear differential equations. Systems of differential 
equations are treated with the help of matrix calculus. Existence and uniqueness theorems 
are proved by Picard’s method of successive approximations, which is also cast in the 
language of contraction operators. 

Part 2 discusses the calculus of functions of several variables. Differential calculus is 
unified and simplified with the aid of linear algebra. It includes chain rules for scalar and 
vector fields, and applications to partial differential equations and extremum problems. 
Integral calculus includes line integrals, multiple integrals, and surface integrals, with 
applications to vector analysis. Here the treatment is along more or less classical lines and 
does not include a formal development of differential forms. 

The special topics treated in Part 3 are Probability and Numerical Analysis. The material 
on probability is divided into two chapters, one dealing with finite or countably infinite 
sample spaces; the other with uncountable sample spaces, random variables, and dis- 
tribution functions. The use of the calculus is illustrated in the study of both one- and 
two-dimensional random variables. 

The last chapter contains an introduction to numerical analysis, the chief emphasis 
being on different kinds of polynomial approximation. Here again the ideas are unified 
by the notation and terminology of linear algebra. The book concludes with a treatment of 
approximate integration formulas, such as Simpson’s rule, and a discussion of Euler’s 
summation formula. 


Vii 


Vill Preface 


There is ample material in this volume for a full year’s course meeting three or four times 
per week. It presupposes a knowledge of one-variable calculus as covered in most first-year 
calculus courses. The author has taught this material in a course with two lectures and two 
recitation periods per week, allowing about ten weeks for each part and omitting the 
starred sections. 

This second volume has been planned so that many chapters can be omitted for a variety 
of shorter courses. For example, the last chapter of each part can be skipped without 
disrupting the continuity of the presentation. Part | by itself provides material for a com- 
bined course in linear algebra and ordinary differential equations. The individual instructor 
can choose topics to suit his needs and preferences by consulting the diagram on the next 
page which shows the logical interdependence of the chapters. 

Once again I acknowledge with pleasure the assistance of many friends and colleagues. 
In preparing the second edition I received valuable help from Professors Herbert S. 
Zuckerman of the University of Washington, and Basil Gordon of the University of 
California, Los Angeles, each of whom suggested a number of improvements. Thanks are 
also due to the staff of Blaisdell Publishing Company for their assistance and cooperation. 

As before, it gives me special pleasure to express my gratitude to my wife for the many 
ways in which she has contributed. In grateful acknowledgement I happily dedicate this 
book to her. 

T. M.A. 
Pasadena, California 
September 16, 1968 
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PART | 
LINEAR ANALYSIS 
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LINEAR SPACES 


1.1 Introduction 


Throughout mathematics we encounter many examples of mathematical objects that 
can be added to each other and multiplied by real numbers. First of all, the real numbers 
themselves are such objects. Other examples are real-valued functions, the complex 
numbers, infinite series, vectors in n-space, and vector-valued functions. In this chapter we 
discuss a general mathematical concept, called a Jinear space, which includes all these 
examples and many others as special cases. 

Briefly, a linear space is a set of elements of any kind on which certain operations (called 
addition and multiplication by numbers) can be performed. In defining a linear space, we 
do not specify the nature of the elements nor do we tell how the operations are to be 
performed on them. Instead, we require that the operations have certain properties which 
we take as axioms for a linear space. We turn now to a detailed description of these axioms. 


1.2 The definition of a linear space 


Let V denote a nonempty set of objects, called elements. The set V is called a linear 
space if it satisfies the following ten axioms which we list in three groups. 


Closure axioms 
AXIOM 1. CLOSURE UNDER ADDITION. For every pair of elements x and y in V there 
corresponds a unique element in V called the sum of x and y, denoted byx + y. 


AXIOM 2. CLOSURE UNDER MULTIPLICATION BY REAL NUMBERS. For every x in V and 
every real number a there corresponds an element in V called the product of a and x, denoted 


by ax. 


Axioms for addition 


AXIOM 3. COMMUTATIVE LAW. For all x and y in V, we havex + y=y+x. 
AXIOM 4. ASSOCIATIVELAW. Forallx, y,andzinV,wehave(x + y)+z=x+(y+zZ). 


3 


4 Linear spaces 
AXIOM 5. EXISTENCE OF ZERO ELEMENT. There is an element in V, denoted by O, such that 
x+O=x ~~ forallxinV. 
AXIOM 6. EXISTENCE OF NEGATIVES. For every x in V, the element (—1)x has the property 


x+(-l)x=0O. 


Axioms for multiplication by numbers 


AXIOM 7. ASSOCIATIVE LAW. For every x in V and all real numbers a and b, we have 
a(bx) = (ab)x. 


AXIOM 8. DISTRIBUTIVE LAW FOR ADDITION IN V. For all x and y in V and all real a, 
we have 


a(x + y) =ax+ay. 


AXIOM 9. DISTRIBUTIVE LAW FOR ADDITION OF NUMBERS. For all x in V and all real 
a and b, we have 
(a + b)x = ax + bx. 


AXIOM 10. EXISTENCE OF IDENTITY. For every x in V, we have 1x = x. 


Linear spaces, as defined above, are sometimes called real linear spaces to emphasize 
the fact that we are multiplying the elements of V by real numbers. If real number is 
replaced by complex number in Axioms 2, 7, 8, and 9, the resulting structure is called a 
complex linear space. Sometimes a linear space is referred to as a linear vector space or 
simply a vector space; the numbers used as multipliers are also called scalars. A real linear 
space has real numbers as scalars; a complex linear space has complex numbers as scalars. 
Although we shall deal primarily with examples of real linear spaces, all the theorems are 
valid for complex linear spaces as well. When we use the term linear space without further 
designation, it is to be understood that the space can be real or complex. 


1.3. Examples of linear spaces 


If we specify the set V and tell how to add its elements and how to multiply them by 
numbers, we get a concrete example of a linear space. The reader can easily verify that 
each of the following examples satisfies all the axioms for a real linear space. 


EXAMPLE 1. Let V = R, the set of all real numbers, and let x + y and ax be ordinary 
addition and multiplication of real numbers. 


EXAMPLE 2. Let V = C, the set of all complex numbers, define x + y to be ordinary 
addition of complex numbers, and define ax to be multiplication of the complex number x 


Examples of linear spaces 5 


by the real number a. Even though the elements of V are complex numbers, this is a real 
linear space because the scalars are real. 


EXAMPLE 3. Let V = V,, the vector space of all n-tuples of real numbers, with addition 
and multiplication by scalars defined in the usual way in terms of components. 


EXAMPLE 4. Let V be the set of all vectors in V,, orthogonal to a given nonzero vector 
N. If n = 2, this linear space is a line through O with N as a normal vector. If n = 3, 
it is a plane through O with N as normal vector. 


The following examples are called function spaces. The elements of V are real-valued 
functions, with addition of two functions f and g defined in the usual way: 


(f+ g)(x) = f(x) + g(*) 


for every real x in the intersection of the domains of fand g. Multiplication of a function 
f by a real scalar a is defined as follows: af is that function whose value at each x in the 
domain of fis af(x). The zero element is the function whose values are everywhere zero. 
The reader can easily verify that each of the following sets is a function space. 


EXAMPLE 5. The set of all functions defined on a given interval. 
EXAMPLE 6. The set of all polynomials. 


EXAMPLE 7. The set of all polynomials of degree <n, where n is fixed. (Whenever we 
consider this set it is understood that the zero polynomial is also included.) The set of 
all polynomials of degree equal to n is not a linear space because the closure axioms are not 
satisfied. For example, the sum of two polynomials of degree n need not have degree n. 


EXAMPLE 8. The set of all functions continuous on a given interval. If the interval is 
[a, b], we denote this space by C(a, b). 


EXAMPLE 9. The set of all functions differentiable at a given point. 
EXAMPLE 10. The set of all functions integrable on a given interval. 


EXAMPLE 11. The set of all functions f defined at 1 with f(1) =0. The number 0 is 
essential in this example. If we replace 0 by a nonzero number c, we violate the closure 
axioms. 


EXAMPLE 12. The set of all solutions of a homogeneous linear differential equation 
y’ + ay + by = 0, where a and b aré given constants. Here again 0 is essential. The set 
of solutions of a nonhomogeneous differential equation does not satisfy the closure axioms. 


These examples and many others illustrate how the linear space concept permeates 
algebra, geometry, and analysis. When a theorem is deduced from the axioms of a linear 
space, we obtain, in one stroke, a result valid for each concrete example. By unifying 
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diverse examples in this way we gain a deeper insight into each. Sometimes special knowl- 
edge of one particular example helps to anticipate or interpret results valid for other 
examples and reveals relationships which might otherwise escape notice. 


1.4 Elementary consequences of the axioms 


The following theorems are easily deduced from the axioms for a linear space. 


THEOREM 1.1. UNIQUENESS OF THE ZERO ELEMENT. Jn any linear space there is one 
and only one zero element. 


Proof. Axiom 5 tells us that there is at least one zero element. Suppose there were two, 
say O, and O,. Taking x = O, and O = O, in Axiom 5, we obtain O, + O, = O,. 
Similarly, taking x = O, and O = O,, we find O, + O, = O,. ButO, + O, = O,+ 0; 
because of the commutative law, so O, = Og. 


THEOREM 1.2. UNIQUENESS OF NEGATIVE ELEMENTS. Jn any linear space every element 
has exactly one negative. That is, for every x there is one and only one y such thatx + y = O. 


Proof. Axiom 6 tells us that each x has at least one negative, namely (—1)x. Suppose 
x has two negatives, say y, and y.. Thenx + y; = Oandx + y, =O. Adding y, to both 
members of the first equation and using Axioms 5, 4, and 3, we find that 


Vo + (x + yi) = yo + O= yp, 
and 
Ytxty)=O2txnty=O+tYy=y+tO=y. 


Therefore y,; = yz, so x has exactly one negative, the element (—1)x. 


Notation. The negative of x is denoted by —x. The difference y — x 1s defined to be 
the sum y + (— x). 


The next theorem describes a number of properties which govern elementary algebraic 
manipulations in a linear space. 


THEOREM 1.3. Ina given linear space, let x and y denote arbitrary elements and let a and b 
denote arbitrary scalars. Then we ‘have the following properties: 

(a) Ox =O. 

(b) a0 =O. 

(c) (—a)x = —(ax) = a(—x). 

(d) If ax = O, then eithera=Oorx =O. 

(ec) [fax =ayanda¥0, thenx=y. 

(f) Ifax = bx and x # O, thena=b. 

(g) —@ + y) = (—x) + (-y) = —x- yy. 

(h) x +x =2x,x +x +x = 3x, and in general, >”, x =nx. 


Exercises is 
We shall prove (a), (b), and (c) and leave the proofs of the other properties as exercises. 


Proof of (a). Let z = 0x. We wish to prove that z = O. Adding z to itself and using 
Axiom 9, we find that 


z+z=0x+0x = (0+ 0)x = 0x =z. 
Now add —z to both members to get z = O. 
Proof of (b). Let z = aO, add z to itself, and use Axiom 8. 
Proof of (c). Let z = (—a)x. Adding z to ax and using Axiom 9, we find that 
z+ax = (—a)x + ax = (-a+a)x =0x =O, 


so z is the negative of ax, z = —(ax). Similarly, if we add a(— x) to ax and use Axiom 8 
and property (b), we find that a(—x) = —(ax). 


1.5 Exercises 


In Exercises 1 through 28, determine whether each of the given sets is a real linear space, if 
addition and multiplication by real scalars are defined in the usual way. For those that are not, 
tell which axioms fail to hold. The functions in Exercises 1 through 17 are real-valued. In Exer- 
cises 3, 4, and 5, each function has domain containing 0 and 1. In Exercises 7 through 12, each 
domain contains all real numbers. 


1. All rational functions. 

2. All rational functions f/g, with the degree of f < the degree of g (including f = 0). 
3. All fwith f(0) = f(1). 8. All even functions. 

4. All fwith 2f(0) = f(1). 9. All odd functions. 

5. All fwith f(l) = 1 + f(0). 10. All bounded functions. 

6. All step functions defined on [0, 1]. 11. All increasing functions. 

7. All f with f(x) > Oas x +0. 12. All functions with period 27. 


13. All f integrable on [0, 1] with J} f(x) dx =0. 

14. All fintegrable on [0, 1] with fj f(x) dx > 0. 

15. All f satisfying f(x) = fC — x) for all x. 

16. All Taylor polynomials of degree < n for a fixed n (including the zero polynomial). 

17. All solutions of a linear second-order homogeneous differential equation y” + P(x)y’ + 
Q(x)y = 0, where P and Q are given functions, continuous everywhere. 

18. All bounded real sequences. 20. All convergent real series. 

19. All convergent real sequences. 21. All absolutely convergent real series. 

22. All vectors (x, y,z) in Vgwith z = 0. 

23. All vectors (x, y, z) in Vg with x =Oory =0. 

24. All vectors (x, y, z) in Vg with y = 5x. 

25. All vectors (x, y, z) in Vgwith 3x +4y =1,2z=0. 

26. All vectors (x, y, z) in V3 which are scalar multiples of (1, 2, 3). 

27. All vectors (x, y, z) in Vg whose components satisfy a system of three linear equations of the 
form: 


QyyX + Apy + aygz = 0, AyX + Agoy + Ag3z = 0, A3)X + Azoy + Ag3z = 0. 
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28. All vectors in V,, that are linear combinations of two given vectors A and B. 

29. Let V = R", the set of positive real numbers. Define the “sum” of two elements x and y in 
V to be their product x - y (in the usual sense), and define ‘‘multiplication’”’ of an element x 
in V by a scalar c to be x°. Prove that V is a real linear space with 1 as the zero element. 

30. (a) Prove that Axiom 10 can be deduced from the other axioms. 

(b) Prove that Axiom 10 cannot be deduced from the other axioms if Axiom 6 is replaced by 
Axiom 6’: For every x in V there is an element y in V such that x + y =O. 

31. Let S be the set of all ordered pairs (x,, x2) of real numbers. In each case determine whether 
or not S is a linear space with the operations of addition and multiplication by scalars defined 
as indicated. If the set is not a linear space, indicate which axioms are violated. 


(a) (x1, X2) + (V1 Yo) = (x, + Vy5 Xe + ye), A(X,, Xg) = (ax,, 0). 
(b) (x1, X9) + (V1, Vo) = (x; + yi, 0), a(x), Xe) = (ax,, AX»). 
(C) (Xy, Xe) + Qi, yo) = (%1, X2 + yo), A(X 1, X_) = (ax,, ax). 


(d) (x1, X2) + (V1, Ve) = (|x, + x2, ly. + Val) s A(X1, Xq) = (lax,|, |axa)). 
32. Prove parts (d) through (h) of Theorem 1.3. 


1.6 Subspaces of a linear space 


Given a linear space V, let S be a nonempty subset of V. If S is also a linear space, with 
the same operations of addition and multiplication by scalars, then S is called a subspace 
of V. The next theorem gives a simple criterion for determining whether or not a subset of 
a linear space is a subspace. 


THEOREM 1.4. Let S be a nonempty subset of a linear space V. Then S is a subspace 
if and only if S satisfies the closure axioms. 


Proof. If S is a subspace, it satisfies all the axioms for a linear space, and hence, in 
particular, it satisfies the closure axioms. 

Now we show that if S satisfies the closure axioms it satisfies the others as well. The 
commutative and associative laws for addition (Axioms 3 and 4) and the axioms for 
multiplication by scalars (Axioms 7 through 10) are dutomatically satisfied in S because 
they hold for all elements of V. It remains to verify Axioms 5 and 6, the existence of a zero 
element in S, and the existence of a negative for each element in S. 

Let x be any element of S. (S has at least one element since S is not empty.) By Axiom 
2, ax is in S for every scalar a. Taking a = 0, it follows that Ox isin S. But Ox = O, by 
Theorem 1.3(a), so O€ S, and Axiom 5 is satisfied. Taking a = —1, we see that (—1)x 
isin S. But x + (—1)x = O since both x and (—1)x are in V, so Axiom 6 1s satisfied in 
S. Therefore S is a subspace of V. 


DEFINITION. Let S be a nonempty subset of a linear space V._ An element x in V of the 


form 
k 


X= > Ox 
i=1 
where x,,...,X,areallin Sand c,,...,c, are scalars, is called a finite linear combination 
of elements of S. The set of all finite linear combinations of elements of S satisfies the closure 
axioms and hence is a subspace of V. We call this the subspace spanned by S, or the linear 
span of S, and denote it by L(S). If S is empty, we define L(S) to be {O}, the set consisting 
of the zero element alone. 
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Different sets may span the same subspace. For example, the space V, is spanned by 
each of the following sets of vectors: {i,j}, {i,j,i + Jj}, {O, i, —i,j, —j,i + J}. The space 
of all polynomials p(t) of degree < n is spanned by the set of n + 1 polynomials 
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It is also spanned by the set {1, ¢/2, 7/3,...,¢"/(a + 1)}, and by {1,1 +4, + 9%,..., 
(1 + 1t)"}. The space of all polynomials is spanned by the infinite set of polynomials 
0a oe eres 

A number of questions arise naturally at this point. For example, which spaces can be 
spanned by a finite set of elements? If a space can be spanned by a finite set of elements, 
what is the smallest number of elements required? To discuss these and related questions, 
we introduce the concepts of dependence, independence, bases, and dimension. These ideas 
were encountered in Volume I in our study of the vector space V,,. Now we extend them 
to general linear spaces. 


1.7 Dependent and independent sets in a linear space 


DEFINITION. A set S of elements in a linear space V is called dependent if there is a finite 
set of distinct elements in S, say X,,...,X,, anda corresponding set of scalars c,,..., Cy; 
not all zero, such that 


An equation > c,x,; = O with not all c, = 0 is said to be a nontrivial representation of O. 
The set S is called independent if it is not dependent. In this case, for all choices of distinct 


elements x,,...,X, in S and scalars c,,..., Cy; 
Ie 
> ¢,x; =O implies ‘c= c. =" =]¢,= 0: 
i=1 


Although dependence and independence are properties of sets of elements, we also apply 
these terms to the elements themselves. For example, the elements in an independent set 
are called independent elements. 

If S is a finite set, the foregoing definition agrees with that given in Volume I for the 
space V,,. However, the present definition is not restricted to finite sets. 


EXAMPLE |. If a subset T of a set S is dependent, then S itself is dependent. This is 
logically equivalent to the statement that every subset of an independent set is independent. 


EXAMPLE 2. If one element in S is a scalar multiple of another, then S is dependent. 
EXAMPLE 3. If O€ S, then S is dependent. 


EXAMPLE 4. The empty set is independent. 
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Many examples of dependent and independent sets of vectors in V,, were discussed in 
Volume I. The following examples illustrate these concepts in function spaces. In each 
case the underlying linear space V is the set of all real-valued functions defined on the real 
line. 


EXAMPLE 5. Let u,(t) = cos? t, up(t) = sin? t, u,(t) = 1 for all real t. The Pythagorean 
identity shows that u, + u, — ug = O, so the three functions u,, up, us are dependent. 


EXAMPLE 6. Let u,(t) = ¢* fork =0,1,2....,andzreal. The set S = {uy,u,,uU2,...} 
is independent. To prove this, it suffices to show that for each n the n + 1 polynomials 
Uy, Uy,..-,U, are independent. A relation of the form > C,U, = O means that 


(1.1) Y c,t* = 0 
k=0 


for all real t. When ¢ = 0, this gives cy =0. Differentiating (1.1) and setting r= 0, 
we find that c, = 0. Repeating the process, we find that each coefficient c, is zero. 


EXAMPLE 7. If a,,..., a, are distinct real numbers, the exponential functions 
u,(x) = e**, ..., u,(x) = et 


are independent. We can prove this by induction on n. The result holds trivially when 
n=1. Therefore, assume it is true for nm — 1 exponential functions and consider scalars 
C1,...,C, such that 


(1.2) > c,e** = 0. 
k=1 


Let ay, be the largest of the nm numbers a,,...,a@,. Multiplying both members of (1.2) 
by e-*™*, we obtain 


(1.3) > ce! 1? = 0, 


k=1 


If k A M, the number a, — aj, 1s negative. Therefore, when x — + 00 in Equation (1.3), 
each term with k # M tends to zero and we find that cy, = 0. Deleting the Mth term from 
(1.2) and applying the induction hypothesis, we find that each of the remaining n — 1 
coefficients c, 1s zero. 


THEOREM 1.5. Let S = {x,,...,X;,} be an independent set consisting of k elements in a 
linear space V and let L(S) be the subspace spanned by S. Then every set of k + 1 elements 
in L(S) is dependent. 


Proof. The proof is by induction on k, the number of elements in S. First suppose 
k =1. Then, by hypothesis, S consists of one element x,, where x, ¥ O since S is 
independent. Now take any two distinct elements y, and y, in L(S). Then each is a scalar 
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multiple of x,, say y, = c,x, and yp = c,x,, where c, and c, are not both 0. Multiplying 
y, by c, and y, by c, and subtracting, we find that 


CoV, — Co = O. 


This is a nontrivial representation of O so y, and y, are dependent. This proves the theorem 
when k = 1. 

Now we assume the theorem is true for k — 1 and prove that it is also true fork. Take 
any set of k + 1 elements in L(S), say T = {y1, Yo, - ++» Veqit. We wish to prove that Tis 
dependent. Since each y, is in L(S) we may write 


(1.4) ); = Saux; 


foreachi=1,2,...,k + 1. Weexamine all the scalars a,, that multiply x, and split the 
proof into two cases according to whether all these scalars are 0 or not. 


CASE 1. a; = 0 for everyi=1,2,...,k +1. In this case the sum in (1.4) does not 
involve x,, so each y, in T is in the linear span of the set S’ = {x,,...,x,}. But S’ is 
independent and consists of k — 1 elements. By the induction hypothesis, the theorem is 
true for k — 1 so the set Tis dependent. This proves the theorem in Case 1. 


CASE 2. Not all the scalars a;; are zero. Let us assume that a,, # 0. (If necessary, we 
can renumber the y’s to achieve this.) Taking i = 1 in Equation (1.4) and multiplying both 
members by c;, where c; = a,;/a,,, we get 


I 
Cy = AjyX1 + D> C41 5%;- 
j=2 


From this we subtract Equation (1.4) to get 
k 

Ciyi — Yi = 2 (cia) — 53)X;, 
j= 


fori =2,...,k +1. This equation expresses each of the k elements c,y, — y; as a linear 
combination of the k — | independent elements x,,...,x;,. By the induction hypothesis, 
the k elements c;y, — y; must be dependent. Hence, for some choice of scalars f,,..., 
thi, not all zero, we have 

k+l 


ZAC, —y) =O, 


from which we find 
k+1 k+1 
(><, Vic dts =O. 


But this is a nontrivial linear combination of y,,..., Yz4; which represents the zero ele- 
ment, so the elements y,,..., y,,, must be dependent. This completes the proof. 
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1.8 Bases and dimension 


DEFINITION. A finite set S of elements in a linear space V is called a finite basis for V 
if S is independent and spans V. The space V is called finite-dimensional if it has a finite 
basis, or if V consists of O alone. Otherwise, V is called infinite-dimensional. 


THEOREM 1.6. Let V be a finite-dimensional linear space. Then every finite basis for V 
has the same number of elements. 


Proof. Let S and T be two finite bases for V. Suppose S consists of k elements and T 
consists of m elements. Since S is independent and spans V, Theorem 1.5 tells us that 
every set of k + 1 elements in Vis dependent. Therefore, every set of more than k elements 
in V is dependent. Since T is an independent set, we must have m < k. The same argu- 
ment with S and T interchanged shows thatk < m. Therefore k = m. 


DEFINITION. Jf a linear space V has a basis of n elements, the integer n is called the 
dimension of V. We writen = dim V. If V = {O}, we say V has dimension 0. 


EXAMPLE |. The space V,, has dimension n. One basis is the set of n unit coordinate 
vectors. 


EXAMPLE 2. The space of all polynomials p(t) of degree < n has dimension n + 1. One 
basis is the set of n + 1 polynomials {1, ¢, t?,...,7¢"}. Every polynomial of degree < nisa 
linear combination of these n + 1 polynomials. 


EXAMPLE 3. The space of solutions of the differential equation y” — 2y’ — 3y = 0 has 
dimension 2. One basis consists of the two functions u,(x) = e*, u(x) = e*”. Every 
solution is a linear combination of these two. 


EXAMPLE 4. The space of all polynomials p(t) is infinite-dimensional. Although the 
infinite set {1, ¢, ¢?,...} spans this space, no finite set of polynomials spans the space. 


THEOREM 1.7. Let V be a finite-dimensional linear space with dim V =n. Then we 
have the following: 

(a) Any set of independent elements in V is a subset of some basis for V. 

(b) Any set of n independent elements is a basis for V. 


Proof. To prove (a), let S = {x,,...,X,} be any independent set of elements in V. 
If L(S) = V, then S is a basis. If not, then there is some element y in V which is not in 
L(S). Adjoin this element to S and consider the new set S’ = {x,,...,,, y}. If this 
set were dependent there would be scalars c,,... , ¢,4,, not all zero, such that 


oo 


CX, + yy =O. 
i=1 


a 


But c,,1 % 0 since x,,..., Xx, are independent. Hence, we could solve this equation for 
y and find that y € L(S), contradicting the fact that y is not in L(S). Therefore, the set S’ 
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is independent but contains k + 1 elements. If L(S’) = V, then S’ is a basis and, since S 
is a subset of S’, part (a) is proved. If S’ is not a basis, we can argue with S’ as we did 
with S, getting a new set S” which contains k + 2 elements and is independent. If S” is a 
basis, then part (a) is proved. If not, we repeat the process. We must arrive at a basis in 
a finite number of steps, otherwise we would eventually obtain an independent set with 
n + 1 elements, contradicting Theorem 1.5. Therefore part (a) is proved. 

To prove (b), let S be any independent set consisting of n elements. By part (a), Sis a 
subset of some basis, say B. But by Theorem 1.6, the basis B has exactly » elements, so 
S= B. 


1.9 Components 


Let V be a linear space of dimension n and consider a basis whose elements e,,..., e, 
are taken in a given order. We denote such an ordered basis as an n-tuple (e,,..., @,). 
If x € V, we can express x as a linear combination of these basis elements: 


(1.5) x= > C;e; « 
i=1 


The coefficients in this equation determine an n-tuple of numbers (c,,...,c,,) that is 
uniquely determined by x. In fact, if we have another representation of x as a linear 
combination of e,,...,@,, say xX = >”, d,e;, then by subtraction from (1.5), we find that 
>", (c; — de; = O. But since the basis elements are independent, this implies c; = d, 
for each i, so we have (c,,...,C¢,) = (dq,,...,d,). 

The components of the ordered n-tuple (c,,...,c¢,) determined by Equation (1.5) are 
called the components of x relative to the ordered basis (e€,,... , €n). 


1.10 Exercises 


In each of Exercises 1 through 10, let S denote the set of all vectors (x, y, z) in Vg whose com- 
ponents satisfy the condition given. Determine whether S is a subspace of V,. If S is a subspace, 
compute dim S. 


lx =0. 6.x =y or x =2 

2.x +y =0. 7.x —y? =0 

3.x +y+2z=0. 8.x +y= 

4.x =y. 9. y=2x and z = 3x. 

3.x =y =Z. 10.x +y+2z=0 and x-y-z=0. 


Let P,, denote the linear space of all real polynomials of degree < n, where n is fixed. In each 
of Exercises 11 through 20, let S denote the set of all polynomials fin P,, satisfying the condition 
given. Determine whether or not S is a subspace of P,,. If S is a subspace, compute dim S. 


11. f(0) =0. 16. f(0) = f(2). 

12. f'(0) =0. 17. fis even. 

13. (0) =0. 18. fis odd. 

14. f(0) + f’(0) =0. 19. fhas degree < k, wherek <n,orf=0. 
15. f(0) =f). 20. fhas degree k, where k <n, orf=0. 


21. In the linear space of all real polynomials p(t), describe the subspace spanned by each of the 
following subsets of polynomials and determine the dimension of this subspace. 


(a) {(1,7,73; (ob {,A,03; © {tA}; dd flt+4(d +97. 
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22. In this exercise, L(S) denotes the subspace spanned by a subset S of a linear space V. Prove 
each of the statements (a) through (f). 
(a) S ¢ L(S). 
(b) If S ¢ T < Vand if Tis a subspace of V, then L(S) ¢ 7. This property is described by 
saying that LCS) is the smallest subspace of V which contains S. 
(c) A subset S of V is a subspace of V if and only if L(S) = S. 
(d) IfS <¢ T¢ V, then L(S) ¢ L(T). 
(e) If S and 7 are subspaces of V, then so is S 1 T. 
(f) If S and 7 are subsets of V, then LUGS A T) ¢ L(S) NO L(T). 
(g) Give an example in which L(S 1 T) # L(S) 0 L(T). 
23. Let V be the linear space consisting of all real-valued functions defined on the real line. 
Determine whether each of the following subsets of V is dependent or independent. Compute 
the dimension of the subspace spanned by each set. 


(a) {1,e%, ec}, a #b. (f) {cos x, sin x}. 

(b) {e%, xe}. (g) {cos? x, sin? x}. 

(Gc) {13.2% ,.xe™*}. (h) {1, cos 2x, sin? x}. 
(d) {e%, xe, x?e%}, (i) {sin x, sin 2x}. 

(e) {e”, e~”, cosh x}. (j) {e* cos x, e~* sin x}. 


24. Let V be a finite-dimensional linear space, and let S be a subspace of V. Prove each of the 
following statements. 
(a) S is finite dimensional and dim S < dim V. 
(b) dim S = dim V if and only if S = V. 
(c) Every basis for S is part of a basis for V. 
(d) A basis for V need not contain a basis for S. 


1.11 Inner products, Euclidean spaces. Norms 


In ordinary Euclidean geometry, those properties that rely on the possibility of measuring 
lengths of line segments and angles between lines are called metric properties. In our study 
of V,,, we defined lengths and angles in terms of the dot product. Now we wish to extend 
these ideas to more general linear spaces. We shall introduce first a generalization of the 
dot product, which we call an inner product, and then define length and angle in terms of the 
inner product. 

The dot product x - y of two vectors x = (x%,,...,x,) and y = (y,,..., Yn) in V, was 
defined in Volume I by the formula 


(1.6) x'y= 2 Ys 


In a general linear space, we write (x, y) instead of x: y for inner products, and we define 
the product axiomatically rather than by a specific formula. That is, we state a number of 
properties we wish inner products to satisfy and we regard these properties as axioms. 


DEFINITION. A real linear space V is said to have an inner product if for each pair of 
elements x and y in V there corresponds a unique real number (x, y) satisfying the following 
axioms for all choices of x, y, z in V and all real scalars c. 


(1) (x,y) = UV, x) (commutativity, or symmetry). 
(2) (x,y +z) = (x, y) + (X, Z) (distributivity, or linearity). 
(3) c(x, y) = (cx, y) (associativity, or homogeneity). 


(4) (x, x) > 0 if x #O (positivity). 
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A real linear space with an inner product 1s called a real Euclidean space. 
Note: Taking c = 0 in (3), we find that (O, y) = 0 for all y. 


In a complex linear space, an inner product (x, y) is a complex number satisfying the 
same axioms as those for a real inner product, except that the symmetry axiom Is replaced 
by the relation 


(1’) (x, y) = (y, x), (Hermitian} symmetry) 


where (y, x) denotes the complex conjugate of (y, x). In the homogeneity axiom, the scalar 
multiplier c can be any complex number. From the homogeneity axiom and (I’), we get 
the companion relation 


(3) (x, cy) = (cy, x) = &(y, x) = (x, y). 


A complex linear space with an inner product is called a complex Euclidean’ space. 
(Sometimes the term unitary space is also used.) One example is complex vector space 
V_,(C) discussed briefly in Section 12.16 of Volume I. 

Although we are interested primarily in examples of real Euclidean spaces, the theorems 
of this chapter are valid for complex Euclidean spaces as well. When we use the term 
Euclidean space without further designation, it is to be understood that the space can be 
real or complex. 

The reader should verify that each of the following satisfies all the axioms for an inner 
product. 


EXAMPLE |. In V,, let (x, y) = x+y, the usual dot product of x and y. 


EXAMPLE 2. If x = (x,, x.) and y = (j,, yg) are any two vectors in V,, define (x, y) by 
the formula 


(x, y) = 2x1 + XVo + X21 + Xoyo- 
This example shows that there may be more than one inner product in a given linear space. 


EXAMPLE 3. Let C(a, b) denote the linear space of all real-valued functions continuous 
on an interval [a, b]. Define an inner product of two functions f and g by the formula 


Cf, 8) = [ f(t)g(t) dt. 


This formula is analogous to Equation (1.6) which defines the dot product of two vectors 
in V,,.. The function values f(t) and g(t) play the role of the components x, and y,, and 
integration takes the place of summation. 


+ In honor of Charles Hermite (1822-1901), a French mathematician who made many contributions to 
algebra and analysis. 
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EXAMPLE 4. In the space C(a, 5), define 


(fa) =] ws(ogcn at, 


where w is a fixed positive function in C(a, b). The function w is called a weight function. 
In Example 3 we have w(t) = 1 for all ¢. 


EXAMPLE 5. In the linear space of all real polynomials, define 


(f,2)= in e'f(t)g(t) dt. 


Because of the exponential factor, this improper integral converges for every choice of 
polynomials f and g. 


THEOREM 1.8. Jn a Euclidean space V, every inner product satisfies the Cauchy-Schwarz 
inequality: 
(x y2< (x), y) = for all x and y in V. 
Moreover, the equality sign holds if and only if x and y are dependent. 
Proof. If either x = O or y = O the result holds trivially, so we can assume that both 
x and y are nonzero. Let z = ax + by, where a and bare scalars to be specified later. We 
have the inequality (z, z) > 0 for allaandb. When we express this inequality in terms of x 


and y with an appropriate choice of a and 5 we will obtain the Cauchy-Schwarz inequality. 
To express (z, z) in terms of x and y we use properties (1°), (2) and (3’) to obtain 


(z, z) = (ax + by, ax + by) = (ax, ax) + (ax, by) + (by, ax) + (by, by) 
= aa(x, x) + ab(x, y) + ba(y, x) + bb(y, y) > 0. 


Taking a = (y, y) and cancelling the positive factor (y, y) in the inequality we obtain 
(y, y)(x, x) + (x, y) + by, x) + 6b > 0. 
Now we take b = —(x, y). Then 6 = —(y, x) and the last inequality simplifies to 
(yy), x) & (%; YQ, ) = 1% yD)? 


This proves the Cauchy-Schwarz inequality. The equality sign holds throughout the proof 
if and only if z = O. This holds, in turn, if and only if x and y are dependent. 


EXAMPLE. Applying Theorem 1.8 to the space C(a, 6) with the inner product (f, g) = 
f° f(g(d) dt , we find that the Cauchy-Schwarz inequality becomes 


({" reco ar)’ < ([?s2@ ar) ([? 0 at). 
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The inner product can be used to introduce the metric concept of length in any Euclidean 
space. 


DEFINITION. Jn a Euclidean space V, the nonnegative number ||x\| defined by the equation 


lx] = @, x)4 


is called the norm of the element x. 


When the Cauchy-Schwarz inequality is expressed in terms of norms, it becomes 


lx, y)| < Ill iy. 


Since it may be possible to define an inner product in many different ways, the norm 
of an element will depend on the choice of inner product. This lack of uniqueness is to be 
expected. It is analogous to the fact that we can assign different numbers to measure the 
length of a given line segment, depending on the choice of scale or unit of measurement. 
The next theorem gives fundamental properties of norms that do not depend on the choice 
of inner product. 


THEOREM 1.9. In a Euclidean space, every norm has the following properties for all 
elements x and y and all scalars c: 

(a) |x| =0 if x=0. 

(b) ||x|| > 0 if x#O (positivity). 

(c) ||cx|| = el ||x|| (homogeneity). 

(d) |x + yl] < xl + yl (triangle inequality). 

The equality sign holds in (d) if x = O, ify = O, or if y = cx for some c> 0. 


Proof. Properties (a), (b) and (c) follow at once from the axioms for an inner product. 
To prove (d), we note that 


|x + yl? = (x +y,x + y) = (x, x) + Uy) + Oy) + OU, ) 
= |x|? + lly? + @, y) +, y).- 


The sum (x, y) + (x, y) is real. The Cauchy-Schwarz inequality shows that |(x, y)| < 
lx] Ilyl] and |(x, y)| < |lxll llyll, so we have 


Ix + ll? < Wl? + lly? + 2 ill yl = Cll + Uy )?. 
This proves (d). When y = cx, where c > 0, we have 


|x + yll = lx + exl| = (1 + ¢) [xl] = [ll + flex] = [ll + ly. 
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DEFINITION. Jn a real Euclidean space V, the angle between two nonzero elements x and 
y is defined to be that number 6 in the interval 0 < 6 < a which satisfies the equation 


(1.7) cos? = veGo) , 
xl yl 


Note: The Cauchy-Schwarz inequality shows that the quotient on the right of (1.7) 
lies in the interval [—1, 1], so there is exactly one @ in [0, 77] whose cosine is equal 
to this quotient. 


1.12 Orthogonality in a Euclidean space 


DEFINITION. Jn a Euclidean space V, two elements x and y are called orthogonal if their 
inner product is zero. A subset S of V is called an orthogonal set if (x, y) = 0 for every pair 
of distinct elements x and y in S. An orthogonal set is called orthonormal if each of its 
elements has norm 1. 


The zero element is orthogonal to every element of V; it is the only element orthogonal to 
itself. The next theorem shows a relation between orthogonality and independence. 


THEOREM 1.10. In a Euclidean space V, every orthogonal set of nonzero elements is 
independent. In particular, in a finite-dimensional Euclidean space with dim V =n, every 
orthogonal set consisting of n nonzero elements is a basis for V. 


Proof. Let S be an orthogonal set of nonzero elements in V, and suppose some finite 
linear combination of elements of S is zero, say 


where each x,€.S. Taking the inner product of each member with x, and using the fact 
that (x,, x,;) = 0 if i 41, we find that c,(x,, x,) = 0. But (x,, x,) ¥ 0 since x, # O so 
c; = 0. Repeating the argument with x, replaced by x;, we find that each c; = 0. This 
proves that S is independent. If dim V = n and if S consists of n elements, Theorem 1.7(b) 
shows that S is a basis for V. 


EXAMPLE. In the real linear space C(0, 277) with the inner product (f, g) = \¢7/f(x)g(x) dx. 
let S be the set of trigonometric functions {u9, u,, Ua, ...} given by 


U(x) = 1, Us, 1(X) = Cosnx, Uy, (Xx) = sin nx, [Ol Ss 2s ess 


If m 4 n, we have the orthogonality relations 


Jy Maun) dx = 0, 
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so Sis an orthogonal set. Since no member of S is the zero element, S is independent. The 
norm of each element of S is easily calculated. We have (ug, uo) = |?” dx = 27 and, for 
n> 1, we have 


27 27 
2 9 
(Us n—1 ’ Us y—1) =| cos° nx dx = 7, (Us, ’ Uy,) =| sin° nx dx = 7. 


Therefore, ||uo|| = Jn and |u,|| = a for n> 1. Dividing each wu, by its norm, we 
obtain an orthonormal set {%, 91, %2,..-} Where y,, = u,/||u,||. Thus, we have 


cos nx sin nx 
aie 5 Po,(X) oe for n>1. 
TT 


V7 - 


P(X) = Pon-(X) = 


= 
Jim 


In Section 1.14 we shall prove that every finite-dimensional Euclidean space has an 
orthogonal basis. The next theorem shows how to compute the components of an element 
relative to such a basis. 


THEOREM 1.11. Let V be a finite-dimensional Euclidean space with dimension n, and 
assume that S = {e,,...,@,} is an orthogonal basis for V. If an element x is expressed as 
a linear combination of the basis elements, say 


i=1 
then its components relative to the ordered basis (e,,..., e,) are given by the formulas 
(1.9) ga oe) for fH 15 2h 
(e; ) é;) 


In particular, if S is an orthonormal basis, each c, is given by 
(1.10) c; = (x, e;). 


Proof. Taking the inner product of each member of (1.8) with e;, we obtain 


(x, e;) = > c(e;, e;) = c,(e;, e;) 


i=l 


since (e,,e;) = Oif i #7. This implies (1.9), and when (e,, e;) = 1, we obtain (1.10). 


If {e,,...,e,} 18 an orthonormal basis, Equation (1.9) can be written in the form 
(1.11) x= > (x, ee; . 
i=1 


The next theorem shows that in a finite-dimensional Euclidean space with an orthonormal 
basis the inner product of two elements can be computed in terms of their components. 
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THEOREM 1.12. Let V be a finite-dimensional Euclidean space of dimension n, and assume 
that {e,,..., &,} is an orthonormal basis for V. Then for every puir of elements x and y in V, 
we have 


(1.12) (x,y) = > (x, e)(y, e;) — (Parseval’s formula). 
i=1 

In particular, when x = y, we have 

(1.13) |x|? = 2 IC e,)|’. 


Proof. Taking the inner product of both members of Equation (1.11) with y and using 
the linearity property of the inner product, we obtain (1.12). When x = y, Equation 
(1.12) reduces to (1.13). 


Note: Equation (1.12) is named in honor of M. A. Parseval (circa 1776-1836), who 
obtained this type of formula in a special function space. Equation (1.13) is a 
generalization of the theorem of Pythagoras. 


1.13. Exercises 


1. Letx = (x,,...,%X,) andy = (y,,..., Yn) bearbitrary vectors in V,. In each case, determine 
whether (x, y) is an inner product for V,, if (x, y) is defined by the formula given. In case 
(x, y) is not an inner product, tell which axioms are not satisfied. 


n n 1/2 
(a) (x,y) = > x4 yal (d) (x,y) = (3 1) : 
t=1 t=1 


(b) (x,y) =| > xy, 
i=l 


(e) (xy) = dy + yd? — D8 — Sy? 
i=1 t=] t=1 


© yy) =d xd yy. 
i=l j=1 


2. Suppose we retain the first three axioms for a real inner product (symmetry, linearity, and 
homogeneity but replace the fourth axiom by a new axiom (4°): (x, x) = 0 if and only if 
x =O. Prove that either (x, x) > 0 for all x # O or else (x, x) < O forall x #O. 


[Hint: Assume (x, x) > 0 for some x # O and (y, y) < 0 for some y # O. In the 
space spanned by {x, y}, find an element z # O with (z, z) = 0.] 


Prove that each of the statements in Exercises 3 through 7 is valid for all elements x and y ina 
real Euclidean space. 


. (x, y) = 0 if and only if ||x + yll = |lx — yl. 

. (x, y) = 0 if and only if |x + yl? = |lxll? + llyll. 

. (x, y) = 0 if and only if |x + cy|| > |x| for all real c. 

(x +y,x — y) = 0 if and only if [xl = |lyll. 

. If x and y are nonzero elements making an angle @ with each other, then 


TAN aA hh W 


lx — yl? = |x|? + llyll? — 2 [xl llyll cos 6. 
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8. In the real linear space C(I, e), define an inner product by the equation 


(fig) = [° dog a fag(x) de. 


(a) If f(x) = Vx, compute || /\). 
(b) Find a linear polynomial g(x) = a + bx that is orthogonal to the constant function 
f(x) = 1. 
9. In the real linear space C(—1, 1), let (4,2) = J1, f(g) dt. Consider the three functions 
Uy, Uz, Ug given by 
u(t) = 1, u(t) =t, u(t) = 1+. 


Prove that two of them are orthogonal, two make an angle z/3 with each other, and two 
make an angle ~/6 with each other. 
10. In the linear space P,, of all real polynomials of degree < n, define 


(f, 8) - r(;) (7) 


(a) Prove that (f, g) is an inner product for P,,. 
(b) Compute (f, g) when f(t) = t and g(t) = at + b. 
(c) If f(t) = ¢, find all linear polynomials g orthogonal to f. 
11. In the linear space of all real polynomials, define (f, g) = f 0 & fg) dt. 
(a) Prove that this improper integral converges absolutely for all polynomials f and g. 
(b) If x,(¢) = forn =0,1,2,..., prove that (x,, Xm) =(m + n)!. 
(c) Compute (f, g) when f(t) = (t + 1 and g(t) = 0 +1. 
(d) Find all linear polynomials g(t) = a + bt orthogonal to f(t) = 1 + ¢. 
12. In the linear space of all real polynomials, determine whether or not (/, g) is an inner product 
if (f, g) is defined by the formula given. In case (/, g) is not an inner product, indicate which 
axioms are violated. In (c), f’ and g’ denote derivatives. 


(a) (fg) = f(g). () (fig) = [fs dt. 
(b) (fg) =| | fg at | , (4) (fg) = ( [fo ar)([*e@ at). 


13. Let V consist of all infinite sequences {x,} of real numbers for which the series 2x? converges. 
If x = {x,} and y = {y,} are two elements of V, define 


(x, y) = >». XnVn- 
n=1 


(a) Prove that this series converges absolutely. 
[Hint: Use the Cauchy-Schwarz inequality to estimate the sum >24, |xnynl.] 


(b) Prove that V is a linear space with (x, y) as an inner product. 
(c) Compute (x, y) if x, = 1/n and y, = 1/(n + 1) forn > 1. 
(d) Compute (x, y) if x, = 2" and y, = I/n! forn > 1. 
14. Let V be the set of all real functions f continuous on [0, + ©) and such that the integral 
Sx e-tf(t) dt converges. Define (f, g) = Jx° e‘f(Og() dt. 
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(a) Prove that the integral for (/, g) converges absolutely for each pair of functions f and g 
in V. 


[Hint: Use the Cauchy-Schwarz inequality to estimate the integral [2/ e~ | f(t)g(2)| dt.] 


(b) Prove that V is a linear space with (f, g) as an inner product. 
(c) Compute (f, g) if f(t) = e~* and g(t) = t", wheren =0,1,2,.... 
15. In a complex Euclidean space, prove that the inner product has the following properties for 
all elements x, y and z, and all complex a and 5. 
(a) (ax, by) = ab(x, y). (b) (x, ay + bz) = a(x, y) + B(x, z). 
16. Prove that the following identities are valid in every Euclidean space. 
(a) Ix + yl? = [xl + Iyl? + @, ») +O. 
(b) |x + yll? — lx — yl? = 2(x, y) + 2, x). 
(c) ix + yl? + llx — yll® = 2 [Lxil? + 2 IlylP. 
17. Prove that the space of all complex-valued functions continuous on an interval [a, b] becomes 
a unitary space if we define an inner product by the formula 


(f.9) = [° wo sog@ at, 


where w is a fixed positive function, continuous on [a, )d]. 


1.14 Construction of orthogonal sets. The Gram-Schmidt process 


Every finite-dimensional linear space has a finite basis. If the space is Euclidean, we can 
always construct an orthogonal basis. This result will be deduced as a consequence of a 
general theorem whose proof shows how to construct orthogonal sets in any Euclidean 
space, finite or infinite dimensional. The construction is called the Gram-Schmidt orthog- 
onalization process, in honor of J. P. Gram (1850-1916) and E. Schmidt (1845-1921). 


THEOREM 1.13. ORTHOGONALIZATION THEOREM. Let x,,%2,..., be a finite or infinite 
sequence of elements in a Euclidean space V, and let L(x,,..., X,) denote the subspace 
spanned by the first k of these elements. Then there is a corresponding sequence of elements 
Y1>¥o5-++,in V which has the following properties for each integer k: 

(a) The element y, is orthogonal to every element in the subspace L(y,,... 5 Ye—1)- 

(b) The subspace spanned by y,,..., ), is the same as that spanned by x1,..., X,! 


Li), , eee » Vie) = L(x,,. °° y Xs 


(c) The sequence y,, y2,..., is unique, except for scalar factors. That is, if y,,Vg,-++» is 
another sequence of elements in V satisfying properties (a) and (b) for all k, then for each k 
there is a scalar c, such that y, = Cyyx- 


Proof. We construct the elements y,, y2,..., by induction. To start the process, we 


take y, = x,. Now assume we have constructed y,,..., y, So that (a) and (b) are satisfied 
when k =r. Then we define y,,, by the equation 


r 
(1.14) Vrti = Xppa 2 yi» 


Construction of orthogonal sets. The Gram-Schmidt process 23 


where the scalars a,,...,a, are to be determined. For j <r, the inner product of y,., 
with y, is given by 


; 
(Vrsi> yJ= (Xe yi) - > aly;, Vi) = Cina y;) — A¥;, V5); 


since (y;, y;) =OifiAj. If y; A O, we can make y,,, orthogonal to y, by taking 
(1.15) igs Ce) 
(53> Ys) 


If y; = O, then y,,, 1s orthogonal to y,; for any choice of a;, and in this case we choose 
a; =O. Thus, the element y,,, is well defined and is orthogonal to each of the earlier 
elements y,,..., y,. Therefore, it is orthogonal to every element in the subspace 


EVs e2d5e)s 


This proves (a) whenk =r+1. 
To prove (b) when kK = r + 1, we must show that L(W,,... , Vpia) = L(%1, ~~~ 5 Xp41)s 
given that L(y,,..., y,) = L(%,,...,%,). The first r elements y,,..., y, are in 


TAK is ei GX) 
and hence they are in the larger subspace L(x,,...,X,,,). The new element y,,, given by 
(1.14) is a difference of two elements in L(x,,..., X,41) so it, too, is in L(x,,..., X41). 
This proves that 
LY; aac > Vrti) = L(x,, ee) Xpit)- 


Equation (1.14) shows that x,,, 1s the sum of two elements in L(j,,..., ¥,41) $0 a similar 
argument gives the inclusion in the other direction: 


Lig 08 ha) S LO 50% 45 pa): 


This proves (b) when k =r +1. Therefore both (a) and (b) are proved by induction on k. 
Finally we prove (c) by induction onk. The case k = | is trivial. Therefore, assume (c) 
is true for k = r and consider the element y,,,. Because of (b), this element is in 


LO isnses Viaa)s 


r+1 
, — — 
Vr4t =» cy; as Zr ee Crt iVr+1 » 
i=l 


SO we can write 


where z,€ L(,,...,),). We wish to prove that z, = O. By property (a), both y_,, and 
CriiVr41 are orthogonal to z,. Therefore, their difference, z,, is orthogonal toz,. In other 
words, z, is orthogonal to itself, soz, = O. This completes the proof of the orthogonaliza- 
tion theorem. 
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In the foregoing construction, suppose we have y,,, = O for some r. Then (1.14) 
shows that x,,, is a linear combination of y,,...,y,, and hence of x,,...,X,, so the 
elements x,,...,X,,; are dependent. In other words, if the first k elements x,,...,x, 
are independent, then the corresponding elements y,,..., y, are nonzero. In this case the 
coefficients a; in (1.14) are given by (1.15), and the formulas defining y,,..., y, become 


(1.16) y,; =x, Vr = Xa — (rst Yi) y for r=1,2,...,k—1. 
i=1 (Vis Vi) 


These formulas describe the Gram-Schmidt process for constructing an orthogonal set of 
nonzero elements y,,..., y, which spans the same subspace as a given independent set 
X,,...,X,.- In particular, if x,,..., x; 1s a basis for a finite-dimensional Euclidean space, 
then y,,..., y;, is an orthogonal basis for the same space. We can also convert this to an 
orthonormal basis by normalizing each element y,, that is, by dividing it by its norm. 
Therefore, as a corollary of Theorem 1.13 we have the following. 


THEOREM 1.14. very finite-dimensional Euclidean space has an orthonormal basis. 


If x and y are elements in a Euclidean space, with y ¥ O, the element 


(x, y) 
(y, y) 4 


is called the projection of x along y. In the Gram-Schmidt process (1.16), we construct 
the element y,,, by subtracting from x,,, the projection of x,,, along each of the earlier 
elements y,,...,y,- Figure 1.1 illustrates the construction geometrically in the vector 
space V3. 


X3 


v3 = X3 — AY, — Agyo, Aj = (x3, i) 


Vi, yi) 


(X2, Yi) 
1, ¥1) 


Ficure 1.1 The Gram-Schmidt process in Vg. An orthogonal set {y,, yo, ys} is 
constructed from a given independent set {x,, x2, x3}. 
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EXAMPLE |. In V4, find an orthonormal basis for the subspace spanned by the three 
vectors x, = (1, —1,1, —1), x, = (5, 1,1, 1), and x, = (—3, —3, 1, —3). 


Solution. Applying the Gram-Schmidt process, we find 


YyrTy1F (1, —1, 1, —1), 


X95 
Yo = Xq (Xs (as Ya) 5 = Xo — yy = (4, 2, 0, 2), 
(Vi> V1) 
xy, Wa, 
yg = xy — Cry, — eI) _ x, — y, + yn = (0,0,0,0). 


(V1, V1) (Yo, Va) 


Since yz; = O, the three vectors x,, xX,, ¥3; must be dependent. But since y, and ys are 
nonzero, the vectors x, and x, are independent. Therefore L(x,, x2, X3) is a subspace of 
dimension 2. The set {y,, y.} is an orthogonal basis for this subspace. Dividing each of 
y, and y, by its norm we get an orthonormal basis consisting of the two vectors 


4 =4(1,-1,1,-1) and —~ ,1,0, 1). 
yal llyell - 50 


EXAMPLE 2. The Legendre polynomials. In the linear space of all polynomials, with the 
inner product (x, y) = J1, x(t)y(t) dt, consider the infinite sequence x9, x,, X2,..., Where 
x,(t) =t"”. When the orthogonalization theorem is applied to this sequence it yields 
another sequence of polynomials yo, y,, ye,..., first encountered by the French mathe- 
matician A. M. Legendre (1752-1833) in his work on potential theory. The first few 
polynomials are easily calculated by the Gram-Schmidt process. First of all, we have 
yo(t) = Xo(t) = 1. Since 


1 1 
Qoyw)=f,dt=2 and (x,y) =| tdt=0, 


we find that 


(x, ’ Yo) 


Ee Yo(t) = x,(t) = t. 


y(t) = x,(t) — 


Next, we use the relations 


1 1 1 
(x2, yo) =. Pdt = 8, (x2, x) = |’ Pdr =0, Onn)=f) tat = 8, 
to obtain 


(Xe, Yo) (t) — (X25 ya) yi) yx(t 


0 ) = t° e: 3° 
(Yo> Yo) (yi, i) 


Ya(t) = X2(t) — 
Similarly, we find that 


ya(t) = 1? — Bt, ya(t) = t* — Ft? + $s, ys(t) = PP — AP + 
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We shall encounter these polynomials again in Chapter 6 in our further study of differential 
equations, and we shall prove that 


n! d”, 
At) = — (t?— 1)”. 
y,At) (on)! dt | ) 
The polynomials P,, given by 
(2n)! 1 d” 4, ‘ 
P(t) = Ap) = —— — 1 =. 1 
“) 2"(n!)° alt) 27n! ar | 


are known as the Legendre polynomials. The polynomials in the corresponding orthonormal 
sequence 9, %1, G2,---, given by ¢, = Yz/||¥nl| are called the normalized Legendre poly- 
nomials. From the formulas for yo,..., ys given above, we find that 


e(N=Vt, oN =V3t, g(t)=WEBPC-1), ot) = IVE (50 — 30), 
y,(t) = 4V3 (35t4 — 3002 + 3), g(t) = AV 42 (632% — 70f? + 157). 


1.15. Orthogonal complements. Projections 


Let V be a Euclidean space and let S be a finite-dimensional subspace. We wish to 
consider the following type of approximation problem: Given an element x in V, to deter- 
mine an element in S whose distance from x is as small as possible. The distance between 
two elements x and y is defined to be the norm ||x — y||. 

Before discussing this problem in its general form, we consider a special case, illustrated 
in Figure 1.2. Here V is the vector space V, and S is a two-dimensional subspace, a plane 
through the origin. Given x in V, the problem is to find, in the plane S, that point s 
nearest to x. 

If xe S, then clearly s = x is the solution. If x is not in S, then the nearest point s 
is obtained by dropping a perpendicular from x to the plane. This simple example suggests 
an approach to the general approximation problem and motivates the discussion that 
follows. 


DEFINITION. Let S be a subset of a Euclidean space V. An element in V is said to be 
orthogonal to S if it is orthogonal to every element of S. The set of all elements orthogonal 
to S is denoted by S* and is called “‘S perpendicular.” 


It is a simple exercise to verify that S+ is a subspace of V, whether or not S itself is one. 
In case S is a subspace, then S- is called the orthogonal complement of S. 


EXAMPLE. If S is a plane through the origin, as shown in Figure 1.2, then S~ is a line 
through the origin perpendicular to this plane. This example also gives a geometric inter- 
pretation for the next theorem. 
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FiGuRE 1.2 Geometric interpretation of the orthogonal decomposition theorem in V3. 


THEOREM 1.15. ORTHOGONAL DECOMPOSITION THEOREM. Let V be a Euclidean space 
and let S be a finite-dimensional subspace of V. Then every element x in V can be represented 
uniquely as a sum of two elements, one in S and one in St. That is, we have 


(1.17) x=s4+st, where sE€S and steS-., 
Moreover, the norm of x is given by the Pythagorean formula 
(1.18) lal]? = [Isl]? + [ls* |. 


Proof. First we prove that an orthogonal decomposition (1.17) actually exists. Since 
Sis finite-dimensional, it has a finite orthonormal basis, say {e,,...,e,}. Given x, define 
the elements s and s+ as follows: 


(1.19) s=Di(x,e)e, st=x-—s. 
i=l 


Note that each term (x, é,)e, is the projection of x along e,;. The element s is the sum of the 
projections of x along each basis element. Since s is a linear combination of the basis 
elements, s lies in S. The definition of s+ shows that Equation (1.17) holds. To prove that 
st lies in S+, we consider the inner product of s+ and any basis element e;. We have 


Ce é;) = (x — S$, e;) = (x, €;) = (s, e;). 


But from (1.19), we find that (s, e;) = (x, e;), so s+ is orthogonal to e;. Therefore s+ 
is orthogonal to every element in S, which means that st € S+. 

Next we prove that the orthogonal decomposition (1.17) is unique. Suppose that x 
has two such representations, say 


(1.20) x=stst and x=t4¢u, 
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where s and tare in S,and s+ and t+ arein St. We wish to prove thats = tand st =¢+. 
From (1.20), we have s — t =f+ — s+, so we need only prove thats—t=O. But 
s—teSandt+ —s+e€S!sos — tis both orthogonal to t+ — s+ and equal tor+ — s+. 
Since the zero element is the only element orthogonal to itself, we must have s — t = O. 
This shows that the decomposition is unique. 

Finally, we prove that the norm of x is given by the Pythagorean formula. We have 


xl? = , x) = +57, 5 +57) = (5,5) + (87,57), 


the remaining terms being zero since s and s+ are orthogonal. This proves (1.18). 


DEFINITION. Let S be a _finite-dimensional subspace of a Euclidean space V, and let 
{e,,...,&,} be an orthonormal basis for S. If x € V, the element s defined by the equation 


s= > (x, ee, 
i=1 
is called the projection of x on the subspace S. 


We prove next that the projection of x on S is the solution to the approximation problem 
stated at the beginning of this section. 


1.16 Best approximation of elements in a Euclidean space by elements in a finite- 
dimensional subspace 


THEOREM 1.16. APPROXIMATION THEOREM. Let S be a finite-dimensional subspace of 
a Euclidean space V, and let x be any element of V. Then the projection of x on S is nearer to 
x than any other element of S. That is, if s is the projection of x on S, we have 


x — sll < llx —¢l 


for all t in S; the equality sign holds if and only if t = s. 


Proof. By Theorem 1.15 we can write x = s+ s+, where se S and s'e¢S+. Then, 
for any ¢ in S, we have 


x—-t=(x-—s)+(s—2). 


Since s — te Sand x —s=s+ eS", this is an orthogonal decomposition of x — t, so 
its norm is given by the Pythagorean formula 


lx — 2]? = |x — sl]? + lls — ell. 


But ||s — ¢||? > 0, so we have ||x — ||? > ||x — s||?, with equality holding if and only if 
s =t. This completes the proof. 
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EXAMPLE |. Approximation of continuous functions on [0, 27] by trigonometric polynomials. 
Let V = C(0, 27), the linear space of all real functions continuous on the interval [0, 27], 
and define an inner product by the equation (/, g) = J27 f(x)g(x) dx. In Section 1.12 
we exhibited an orthonormal set of trigonometric functions @, 91, Y2,..., Where 


1 cos kx sin kx 
1.21 A S= xX) = 1; xj= me 16 for k>1. 
( ) Pol ) an Por 1(X) re Parl ) we za 


The 2n + 1 elements 9, 91,---, Ye, Span a subspace S of dimension 27 + 1. The ele- 
ments of S are called trigonometric polynomials. 
If fe C(O, 27), let f,, denote the projection of f on the subspace S. Then we have 


(1.22) fr = SUL edn» — where (fg) = [" Foden) de. 


The numbers (f, y,) are called Fourier coefficients of f. Using the formulas in (1.21), we 
can rewrite (1.22) in the form 


(1.23) f(x) = 4a) + d (a, cos kx + b;, sin kx), 
k=1 
where 


27 20 
a, = - f(x) cos kx dx, b, = u f(x) sin kx dx 
TT J0 TT /J90 


fork =0,1,2,...,2. The approximation theorem tells us that the trigonometric poly- 
nomial in (1.23) approximates f better than any other trigonometric polynomial in S, in 
the sense that the norm || f —_/,,|| is as small as possible. 


EXAMPLE 2. Approximation of continuous functions on [—1,1] by polynomials of 
degree <n. Let V = C(—1, 1), the space of real continuous functions on [—1, 1], and let 
Cf, 2g) = Jt, fOdg(x) dx. The n + 1 normalized Legendre polynomials 9, 91,..-, Pn; 
introduced in Section 1.14, span a subspace S of dimension n + | consisting of all poly- 
nomials of degree <n. If fe C(—1, 1), let f, denote the projection of fon S. Then we 
have 


Jn =D Pr) Pre» where (f, Fe) = [ 00.0) dt. 


This is the polynomial of degree < 1 for which the norm || f — /,|| is smallest. For example, 
when f(x) = sin 7x, the coefficients (/, y,) are given by 


(f, P) = le sin at y,(t) dt. 


In particular, we have (/, %) = 0 and 


1 13 32 
, 9) = ~tsinwtdt= /==. 
(f, V1) {i 7 ja? 


30 


Linear spaces 


Therefore the linear polynomial f,(t) which is nearest to sin wt on [—1, 1] is 


f(t) = /22 p(t) = 


3 


=t. 
1 


Since (f, g2) = 0, this is also the nearest quadratic approximation. 


1.17 Exercises 


1. 


In each case, find an orthonormal basis for the subspace of V3 spanned by the given vectors. 
(a) x, =(,1, 1), xX, = (1,0, 1), x3 = (3, 2, 3). 
(b) x,=(,1, 1), 43 = (1,1, =I), x3 = (1,0, 1). 


. In each case, find an orthonormal basis for the subspace of V, spanned by the given vectors. 


(a) x, = (1,1, 0,9), x, = (0,1, 1,90), x3 = (0,0, 1, 1), x, = (1,0, 0, 1). 
(b) x, = (1, 1,0, 1), x, = (1, 9, 2, 1), x3 = (1, 2, =2, 1). 


. In the real linear space C(O, 7), with inner product (x, y) = f i x(t)y(t) dt, let x,(t) = cos nt 


forn =0,1,2,.... Prove that the functions yo, yy, yo,-..., given by 


1 2 
yo(t) = we and Yn(t) -/7 Cos nt for n>, 


form an orthonormal set spanning the same subspace as x9, X,,%,..-. 


. In the linear space of all real polynomials, with inner product (x, y) = fa x(t)y(t) dt, let 


x,(t) = ¢t” forn =0,1,2,.... Prove that the functions 


y(t)=1, y(t) = V3Qt—-1), y(t) = V5 (62 — 6 + 1) 


form an orthonormal set spanning the same subspace as {xy, x1, Xo}. 


. Let V be the linear space of all real functions f continuous on [0, +) and such that the 


integral {5° e-‘f(t) dt converges. Define (f, 2) = {f° e‘f(Og(d dt, and let yo, v1, ye,..., be 
the set obtained by applying the Gram-Schmidt process to x9, x,, x2,..., where x,(t) = 2” 
for n>0. Prove that y(t) =1, y,(@) =t -—1, y(t) = -—4t4+2, y(t) = -—9P + 
18t — 6. 


. In the real linear space C(1, 3) with inner product (f, 2) = fi ff (x)g(x) dx, let f(x) = 1/x 


and show that the constant polynomial g nearest to fis g = $ log 3. Compute ||g — f ||? for 
this g. 


. In the real linear space C(O, 2) with inner product (f, g) = f2 ff (x)g(x) dx, let f(x) = e* and 


show that the constant polynomial g nearest to fis g = $(e? — 1). Compute ||g — f ||? for 
this g. 


. In the real linear space C(—1, 1) with inner product (f,g) = J1, f()g(x) dx, let f(x) = e* 


and find the linear polynomial g nearest to f. Compute ||g — / ||? for this g. 


. In the real linear space C(O, 27) with inner product (f, 2) = |2” f(x)g(x) dx, let f(x) = x. 


In the subspace spanned by u(x) = 1, u,(x) = cos x, uo(x) = sin x, find the trigonometric 
polynomial nearest to f- 


. In the linear space V of Exercise 5, let f(x) = e~* and find the linear polynomial that is nearest 


tof. 
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LINEAR TRANSFORMATIONS AND MATRICES 


2.1 Linear transformations 


One of the ultimate goals of analysis is a comprehensive study of functions whose 
domains and ranges are subsets of linear spaces. Such functions are called transformations, 
mappings, or operators. This chapter treats the simplest examples, called Jinear transforma- 
tions, which occur in all branches of mathematics. Properties of more general transforma- 
tions are often obtained by approximating them by linear transformations. 

First we introduce some notation and terminology concerning arbitrary functions. Let 
V and W be two sets. The symbol 


T: V>Ww 


will be used to indicate that Tis a function whose domain is V and whose values are in W. 
For each x in V, the element 7(x) in W is called the image of x under T, and we say that T 
maps x onto T(x). If A is any subset of V, the set of all images 7(x) for x in A is called the 
image of A under T and is denoted by 7(A). The image of the domain V, 7(V), is the range 


of T. 
Now we assume that V and W are linear spaces having the same set of scalars, and we 


define a linear transformation as follows. 


DEFINITION. Jf V and W are linear spaces, a function T: V — W is called a linear trans- 
formation of V into W if it has the following two properties: 

(a) Tix+y)=T(x)+T7(y) forall x and y in V, 

(b) T(cx) = cT(x) for all x in V and all scalars c. 


These properties are verbalized by saying that T preserves addition and multiplication by 
scalars. The two properties can be combined into one formula which states that 


T(ax + by) = aT(x) + bT(y) 
for all x,y in Vand all scalars aand b. By induction, we also have the more general relation 
T ( >» ax = > 4;T(%,) 
i=1 i=1 


for any n elements x,,..., x, in V and any n scalars q,..., ay. 
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The reader can easily verify that the following examples are linear transformations. 


EXAMPLE 1. The identity transformation. The transformation T: V— V, where 7(x) = x 
for each x in V, is called the identity transformation and is denoted by J or by J. 


EXAMPLE 2. The zero transformation. The transformation T: V—» V which maps each 
element of V onto O is called the zero transformation and is denoted by O. 


EXAMPLE 3. Multiplication by a fixed scalar c. Here we have T: V ~ V, where T(x) = cx 
for all x in V. When c = 1, this is the identity transformation. When c = 0, itis the zero 


transformation. 


EXAMPLE 4, Linear equations. Let V = V, and W= V,,. Given mn real numbers a;,, 


wherei=1,2,...,mandk =1,2,...,n, define T: V, — V,, as follows: T maps each 
vector x = (x,,...,X,) in V,, onto the vector y = (j,,..-., Ym) in V,, according to the 
equations 
¥,= > a,x, for i=1,2,...,m. 
k=1 


EXAMPLE 5. Inner product with a fixed element. Let V be a real Euclidean space. For a 
fixed element z in V, define T: V > R as follows: If xe V, then T(x) = (x, z), the inner 
product of x with z. 


EXAMPLE 6. Projection on a subspace. Let V be a Euclidean space and let S be a finite- 
dimensional subspace of V. Define T: VS as follows: If xe V, then T(x) is the 
projection of x on S. 


EXAMPLE 7. The differentiation operator. Let V be the linear space of all real functions 
f differentiable on an open interval (a,b). The linear transformation which maps each 
function fin V onto its derivative f” is called the differentiation operator and is denoted by 
D. Thus, we have D: V > W, where D(f) =f" for each fin V. The space W consists of 
all derivatives f’. 


EXAMPLE 8. The integration operator. Let V be the linear space of all real functions 
continuous on an interval [a, b]. If fe V, define g = T(/) to be that function in V given by 


a=] f(Qd if a<x<b. 
This transformation 7 is called the integration operator. 


2.2 Null space and range 


In this section, T denotes a linear transformation of a linear space V into a linear space W. 


THEOREM 2.1. The set T(V) (the range of T) is a subspace of W. Moreover, T maps 
the zero element of V onto the zero element of W. 
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Proof. To prove that T(V) is a subspace of W, we need only verify the closure axioms. 
Take any two elements of 7(V), say T(x) and T(y). Then T(x) + T(y) = T(x + y), so 
T(x) + T(y) isin T(V). Also, for any scalar c we have cT(x) = T(cx), so cT(x) is in T(V). 
Therefore, T(V) is a subspace of W. Taking c = 0 in the relation T(cx) = cT(x), we find 
that 7(O) = O. 


DEFINITION. The set of all elements in V that T maps onto O is called the null space of 
T and is denoted by N(T). Thus, we have 


N(T) = {x|xeV and T(x) = O}. 
The null space is sometimes called the kernel of T. 
THEOREM 2.2. The null space of T is a subspace of V. 
Proof. If x and y are in N(7), then so are x + y and cx for all scalars c, since 
T(x + y)=T(x)+ TQ) =O and Tcx) = cl (x): = 0; 


The following examples describe the null spaces of the linear transformations given in 
Section 2.1. 


EXAMPLE |. Identity transformation. The null space is {O}, the subspace consisting of 
the zero element alone. 


EXAMPLE 2. Zero transformation. Since every element of V is mapped onto zero, the 
null space is V itself. 


EXAMPLE 3. Multiplication by a fixed scalar c. If c #0, the null space contains only O. 
If c = 0, the null space is V. 


EXAMPLE 4. Linear equations. The null space consists of all vectors (x,,...,%X,) in V, 
for which 
n 
> aX, = 0 for i=1,2,...,m. 
k=1 


EXAMPLE 5. Inner product with a fixed element z. The null space consists of all elements 
in V orthogonal to z. 


EXAMPLE 6. Projection on a subspace S. If x eV, we have the unique orthogonal 
decomposition x = s+ s+ (by Theorem 1.15). Since T(x) =s5, we have T(x) =O 


if and only if x = s+. Therefore, the null space is S+, the orthogonal complement of S. 


EXAMPLE 7. Differentiation operator. The null space consists of all functions that are 
constant on the given interval. 


EXAMPLE 8. Integration operator. The null space contains only the zero function. 
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2.3 Nullity and rank 


Again in this section T denotes a linear transformation of a linear space V into a linear 
space W. We are interested in the relation between the dimensionality of V, of the null 
space N(T), and of the range T(V). If V is finite-dimensional, then the null space is also 
finite-dimensional since it is a subspace of V. The dimension of N(T) is called the nullity 
of T. In the next theorem, we prove that the range is also finite-dimensional; its dimension 
is called the rank of T. 


THEOREM 2.3. NULLITY PLUS RANK THEOREM. If V is finite-dimensional, then T(V) is 
also finite-dimensional, and we have 


(2.1) dim N(T) + dim T(V) = dim V. 


In other words, the nullity plus the rank of a linear transformation is equal to the dimension 
of its domain. 


Proof. Letn = dim Vand lete,,...,e, bea basis for N(T), where k = dim N(T) < n. 
By Theorem 1.7, these elements are part of some basis for V, say the basis 


(2.2) C1960 9 Ons Crizy e+ 5 Cxins 


where k +r =n. We shall prove that the r elements 
(2.3) T(Cpst)s +++ > Terr) 


form a basis for 7(V), thus proving that dim 7(V) = r. Since k + r = 7, this also proves 
(2.1). 

First we show that the r elements in (2.3) span 7(V). If ye T(V), we have y = T(x) 
for some x in V, and we can write x = cyey +°°* + Cy4,€,4,- Hence, we have 


k+r k k+r k+r 


y = T(x) = 2.«T(e) = 2 iT (6) 5 a 2 Ted) = > CT) 


since T(e,) = +“: = T(e,) = O. This shows that the elements in (2.3) span T(V). 
Now we show that these elements are independent. Suppose that there are scalars 


Cysts +++ 5 Cys, Such that 


k+r 


> ¢T(e) =O. 


i=k+1 
This implies that 


k+r 
r( > ei) = O 


i=k+1 


so the element x = Cyi@pia Ht H+ Cys Cpy, IS In the null space N(7). This means there 
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are scalars c,,..., C, Such that x = cje; +--+ + c,e,, So we have 


k k+r 
x—-x=Ycee,— > ce, =O. 
t=1 t=k+1 


But since the elements in (2.2) are independent, this implies that all the scalars c, are zero. 
Therefore, the elements in (2.3) are independent. 


Note: If V is infinite-dimensional, then at least one of N(T) or T(V) is infinite- 
dimensional. A proof of of this fact is outlined in Exercise 30 of Section 2.4. 


2.4 Exercises 


In each of Exercises 1 through 10, a transformation T: V,— V, is defined by the formula given 
for T(x, y), where (x, y) is an arbitrary point in V,. In each case determine whether T is linear. If 
T is linear, describe its null space and range, and compute its nullity and rank. 


1. T(x, y) = (,). 6. T(x, y) = (e”, e”). 

2. T(x, y) = (x, —y). 7. T(x, y) = (x, 1). 

3. T(x, y) = (x, 0). 8. Tix,y=(x+1,y +1). 
4. T(x, y) = (x, x). 9. T(Xx,y)=(* -—y,x +y). 
5. T(x, y) = @*, y*). 10. T(x, y) = (2x — y,x + y). 


Do the same as above for each of Exercises 11 through 15 if the transformation T: V,— V, 
is described as indicated. 


11. T rotates every point through the same angle ¢ about the origin. That is, T maps a point 
with polar coordinates (r, 6) onto the point with polar coordinates (r, 8 + 9), where ¢ is 
fixed. Also, T maps O onto itself. 

12. T maps each point onto its reflection with respect to a fixed line through the origin. 

13. T maps every point onto the point (1, 1). 

14. T maps each point with polar coordinates (r, 9) onto the point with polar coordinates (2r, 6). 
Also, T maps O onto itself. 

15. T maps each point with polar coordinates (r, @) onto the point with polar coordinates (r, 29). 
Also, T maps O onto itself. 


Do the same as above in each of Exercises 16 through 23 if a transformation T: V3— V3 is 
defined by the formula given for T(x, y, z), where (x, y, z) is an arbitrary point of V3. 


16. T(x, y, z) = (Z, y, x). 20. T(x, y,z) =(x + 1,y +1,z —-1). 
17. T(x, y, z) = (x, y, 0). 21 E32) =O ly 252-4 3). 
18. T(x, y, z) = (x, 2y, 3z). 22. T(x; ye2) =X "5 2") 

19. T(x, y, Z) = (x, y, 1). 23. T(x, y,zZ) =(x +2,0,x + y). 


In each of Exercises 24 through 27, a transformation T: V—» V is described as indicated. In 
each case, determine whether T is linear. If 7 is linear, describe its null space and range, and 
compute the nullity and rank when they are finite. 


24. Let V be the linear space of all real polynomials p(x) of degree <n. If p€ V,q = T(p) means 
that g(x) = p(x + 1) for all real x. 

25. Let V be the linear space of all real functions differentiable on the open interval (—1, 1). 
If fe V,g = T(f) means that g(x) = xf’(x) for all x in (—1, 1). 
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26. 


27. 


28. 


29. 


30. 
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Let V be the linear space of all real functions continuous on [a, b]. If fe V, g = T(f/) means 
that 


&(x) =| fo sin (x — t)adt for a<x<b. 


Let V be the space of all real functions twice differentiable on an open interval (a, b). If 
y€V, define T(y) = y” + Py’ + Qy, where P and Q are fixed constants. 

Let V be the linear space of all real convergent sequences {x,}. Define a transformation 
T: V— V as follows: If x = {x,} is a convergent sequence with limit a, let T(x) = {yn}, 
where y, = @ — x,forn > 1. Prove that Tis linear and describe the null space and range of T. 
Let V denote the linear space of all real functions continuous on the interval [—7z, 7]. Let S 
be that subset of V consisting of all f satisfying the three equations 


[" fat =o, f° f(t) cost dt =0, {i fd) sin tdt =0. 


(a) Prove that S is a subspace of V. 
(b) Prove that S contains the functions f(x) = cos nx and f(x) = sin nx foreachn = 2,3,.... 
(c) Prove that S is infinite-dimensional. 

Let T: V—» V be the linear transformation defined as follows: If fe V,g = T(f) means that 


gx) =|" {1 +cos (x — n}f(O dt. 


(d) Prove that T(V), the range of T, is finite-dimensional and find a basis for T(V). 

(e) Determine the null space of T. 

(f) Find all real c 4 0 and all nonzero f in V such that T(f) = cf. (Note that such an f 
lies in the range of 7.) 

Let T: V—» W be a linear transformation of a linear space V into a linear space W. If V is 
infinite-dimensional, prove that at least one of T(V) or N(7T) is infinite-dimensional. 


[Hint: Assume dim N(T) =k, dim T(V) =r, lete,,...,e, be a basis for N(T) and 
let €),..., Cx, Cxi3,+++sexin be independent elements in V, where n>r. The 
elements 7T(é,,;),--., T(€zin) are dependent since n >r. Use this fact to obtain a 
contradiction. ] 


2.5 Algebraic operations on linear transformations 


Functions whose values lie in a given linear space W can be added to each other and can 


be multiplied by the scalars in W according to the following definition. 


DEFINITION. Let S: V—> W and T: V — W be two functions with a common domain V 


and with values in a linear space W. If c is any scalar in W, we define the sum S + T and the 
product cT by the equations 


(2.4) (S+T)(x)=S@+7TR), (CTI) = eT) 


forall xin V. 
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We are especially interested in the case where V is also a linear space having the same 
scalars as W. In this case we denote by #(V, W) the set of all linear transformations of V 
into W. 

If S and 7 are two linear transformations in #(V, W), it is an easy exercise to verify that 
S + T and cT are also linear transformations in “(V, W). More than this is true. With 
the operations just defined, the set “(V, W) itself becomes a new linear space. The zero 
transformation serves as the zero element of this space, and the transformation (—1)T 
is the negative of T. It is a straightforward matter to verify that all ten axioms for a linear 
space are satisfied. Therefore, we have the following. 


THEOREM 2.4. The set £(V, W) of all linear transformations of V into W is a linear 
space with the operations of addition and multiplication by scalars defined as in (2.4). 


A more interesting algebraic operation on linear transformations is composition or 
multiplication of transformations. This operation makes no use of the algebraic structure 
of a linear space and can be defined quite generally as follows. 


Ficure 2.1 Illustrating the composition of two transformations. 


DEFINITION. Let U, V, W be sets. Let T: U-> V be a function with domain U and 
values in V, and let S: V—> W be another function with domain V and values in W. Then 
the composition ST is the function ST: U — W defined by the equation 


(ST)(x) = S[T(x)] —_ for every x in U. 


Thus, to map x by the composition ST, we first map x by T and then map T(x) by S. 
This is illustrated in Figure 2.1. 

Composition of real-valued functions has been encountered repeatedly in our study of 
calculus, and we have seen that the operation is, in general, not commutative. However, 
as in the case of real-valued functions, composition does satisfy an associative law. 


THEOREM 2.5. IfT: U>V,S:V—W,and R: W— X are three functions, then we have 


R(ST) = (RS)T. 
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Proof. Both functions R(ST) and (RS)T have domain U and values in X. For each x 
in U, we have 


[R(ST)](x) = RE(ST)@)] = RIS[TQ)]] and [(RS)T](x) = (RS)[T(x)] = RISITO@)], 


which proves that R(ST) = (RS)T. 


DEFINITION. Let T: V-> V be a function which maps V into itself. We define integral 
powers of T inductively as follows: 


PST, T*?=T7T"" for n>1. 
Here / is the identity transformation. The reader may verify that the associative law 


implies the law of exponents 7”T” = T™*” for all nonnegative integers m and n. 
The next theorem shows that the composition of Jinear transformations is again linear. 


THEOREM 2.6. If U, V, W are linear spaces with the same scalars, and if T: U>V 
and S: V — W are linear transformations, then the composition ST: U — W is linear. 


Proof. For all x, y in U and all scalars a and b, we have 
(ST)(ax + by) = S[T(ax + by)] = S[aT(x) + bT()] = aST(x) + bST(y). 


Composition can be combined with the algebraic operations of addition and multiplica- 
tion of scalars in “(V, W) to give us the following. 


THEOREM 2.7. Let U, V, W be linear spaces with the same scalars, assume S and T are 
in £(V, W), and let c be any scalar. 
(a) For any function R with values in V, we have 


(S+7)R=SR+T7R and (cS)R = c(SR). 
(b) For any linear transformation R: W— U, we have 


R(S+7)=RS+ RT and R(cS)=c(RS). 


The proof is a straightforward application of the definition of composition and is left as 
an exercise. 


2.6 Inverses 


In our study of real-valued functions we learned how to construct new functions by 
inversion of monotonic functions. Now we wish to extend the process of inversion to a 
more general class of functions. 
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Given a function T, our goal is to find, if possible, another function S whose composition 
with T is the identity transformation. Since composition is in general not commutative, 
we have to distinguish between ST and TS. Therefore we introduce two kinds of inverses 
which we call left and right inverses. 


DEFINITION. Given two sets V and W anda functionT: VW. A function S: TV)— V 
is called a left inverse of T if S[T(x)] = x for all x in V, that is, if 


ST = ly, 


where I is the identity transformation on V, A function R: T(V) — V is called a right inverse 
of T if T[R(y)] = y for all y in T(V), that is, if 


TR =I y)> 
where I y) is the identity transformation on T(V). 


EXAMPLE. A function with no left inverse but with two right inverses. Let V = {1, 2} 
and let W = {0}. Define 7: V—> Was follows: T(1) = 7(2) = 0. This function has two 
right inverses R: W— V and R’: W— V given by 


RO)=1, £R (0) =2. 
It cannot have a left inverse S since this would require 
= S[71)]} = S(0) and 2=S[T(2)] = S(O). 


This simple example shows that left inverses need not exist and that right inverses need not 
be unique. 


Every function T: V — W has at least one right inverse. In fact, each y in T(V) has the 
form y = 7(x) for at least one x in V. If we select one such x and define R(y) = x, then 
T[R(y)] = T(x) = y for each y in T(V), so R is a right inverse. Nonuniqueness may occur 
because there may be more than one x in V which maps onto a given yin T(V). We shall 
prove presently (in Theorem 2.9) that if each y in T(V) is the image of exactly one x in V, 
then right inverses are unique. 

First we prove that if a left inverse exists it is unique and, at the same time, is a right 
inverse. 


THEOREM 2.8. A function T: V—» W can have at most one left inverse. If T has a left 
inverse S, then S is also a right inverse. 


Proof. Assume T has two left inverses, S: T(V) > Vand S’: T(V) > V. Choose any 
yin T(V). We shall prove that S(y) = S’(y). Now y = T(x) for some x in V, so we have 


S[T(x)] = x and S’[T(x)] = x, 
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since both S and S’ are left inverses. Therefore S(y) = x and S’(y) = x, so S(y) = S’()) 
for all yin T7(V). Therefore S = S” which proves that left inverses are unique. 

Now we prove that every left inverse S is also a right inverse. Choose any element y in 
T(V). We shall prove that T[S(y)] = y. Since ye T(V), we have y = T(x) for some x in 
V. But Sis a left inverse, so 


x = S[T(x)] = SQ). 


Applying T, we get T(x) = T[S(y))]. But y = T(x), so y = T[S(y)], which completes the 
proof. 


The next theorem characterizes all functions having left inverses. 


THEOREM 2.9. A function T: V— W has a left inverse if and only if T maps distinct 
elements of V onto distinct elements of W; that is, if and only if, for all x and y in V, 


(2.5) xy implies T(x) € TV). 
Note: Condition (2.5) is equivalent to the statement 
(2.6) T(x) = TY) implies x=y. 
A function T satisfying (2.5) or (2.6) for all x and y in V is said to be one-to-one on V. 


Proof. Assume T has a left inverse S, and assume that T(x) = T(y). We wish to prove 
that x = y. Applying S, we find S[7(x)] = S[T(y)]. Since S[T(x)] = x and S[T())] = y, 
this implies x = y. This proves that a function with a left inverse is one-to-one on its 
domain. 

Now we prove the converse. Assume J is one-to-one on V. We shall exhibit a function 
S: T(V) — V which is a left inverse of T. If ye T(V), then y = T(x) for some xin V. By 
(2.6), there is exactly one x in V for which y = T(x). Define S(y) to be this x. That is, 
we define S on T(V) as follows: 


SQ) =x  meansthat T(x)=y. 


Then we have S[7(x)] = x for each x in V, so ST = J. Therefore, the function S so 
defined is a left inverse of T. 


DEFINITION. Let T: V— W be one-to-one on V. The unique left inverse of T (which 
we know is also a right inverse) is denoted by T~1. We say that T is invertible, and we call 
T—! the inverse of T. 


The results of this section refer to arbitrary functions. Now we apply these ideas to 
linear transformations. 
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2.7 One-to-one linear transformations 


In this section, V and W denote linear spaces with the same scalars, and T: V+ W 
denotes a linear transformation in #(V, W). The linearity of T enables us to express the 
one-to-one property in several equivalent forms. 


THEOREM 2.10. Let T: V—> W bea linear transformation in £(V, W). Then the following 
statements are equivalent. 
(a) T is one-to-one on V. 
(b) Tis invertible and its inverse T-!: T(V) — V is linear. 
(c) For all x in V, T(x) = O implies x = O. That is, the null space N(T) contains only 
the zero element of V. 


Proof. We shall prove that (a) implies (b), (b) implies (c), and (c) implies (a). First 
assume (a) holds. Then T has an inverse (by Theorem 2.9), and we must show that T~} 
is linear. Take any two elements u and vin 7(V). Then u = T(x) and v = T(y) for some 
x and yin V. For any scalars a and b, we have 


au + bv = aT(x) + bT(y) = T(ax + by), 
since Tis linear. Hence, applying 71, we have 
T-(au + bv) = ax + by = aT" (u) + ODT (0), 


so T~1is linear. Therefore (a) implies (b). 

Next assume that (b) holds. Take any x in V for which T(x) = O. Applying T~1, we 
find that x = T~\(O) = O, since T~1 is linear. Therefore, (b) implies (c). 

Finally, assume (c) holds. Take any two elements u and v in V with J(u) = T(v). By 
linearity, we have T(u — v) = T(u) — T(v) = O,sou — v = O. Therefore, Tis one-to-one 
on V, and the proof of the theorem is complete. 


When V is finite-dimensional, the one-to-one property can be formulated in terms of 
independence and dimensionality, as indicated by the next theorem. 


THEOREM 2.11. Let T: V— W be a linear transformation in £(V, W) and assume that 
V is finite-dimensional, say dim V =n. Then the following statements are equivalent. 
(a) T is one-to-one on V. 
(b) If e,,...,e, are independent elements in V, then T(e,),..., T(€,) are independent 
elements in T(V). 
(c) dim T(V) =n. 
(d) If {e,,...,e,} is a basis for V, then {T(e,),..., T(e,)} is a basis for T(V). 


Proof. We shall prove that (a) implies (b), (b) implies (c), (c) implies (d), and (d) implies 
(a). Assume (a) holds. Let e,,...,e, be independent elements of V and consider the 


42 Linear transformations and matrices 


elements T(e,),..., T(e,) in T(V). Suppose that 
p 
> c;,T(e,) =O 
i=1 
for certain scalars c,,...,¢,. By linearity, we obtain 
Dp p 
(2 ce =O, and hence > ce=0 
i=1 i=1 


since Tis one-to-one. But e,,..., e, are independent, soc, =:::=c, =0. Therefore 
(a) implies (b). 

Now assume (b) holds. Let {e,,...,e,} be a basis for V. By (b), the n elements 
T(e,),.-+, T(€,) in T(V) are independent. Therefore, dim T(V) >n. But, by Theorem 
2.3, we have dim T(V) <n. Therefore dim T(V) = 7, so (b) implies (c). 

Next, assume (c) holds and let {e,,...,e,} be a basis for V. Take any element y in 
T(V). Then y = 7(x) for some x in V, so we have 


x=)Dce,, andhence y= T(x) = > ¢,T(e,). 
i=1 


i=l 
Therefore {7(e,),..., T(e,)} spans 7(V). But we are assuming dim 7(V)=n, so 
{T(e,),..., T(e,)} is a basis for T(V). Therefore (c) implies (d). 
Finally, assume (d) holds. We will prove that T(x) = Oimpliesx = O. Let {e,,...,e,} 
be a basis for V. If x € V, we may write 


x=)>ce,, andhence T(x) = > ¢;,T(e,). 
i=1 i=l 


If T(x) = O,thenc, =:+-- =c, = 0, since the elements 7(e),..., T(e,) are independent. 
Therefore x = O, so Tis one-to-one on V. Thus, (d) implies (a) and the proof is complete. 


2.8 Exercises 


1. Let V = {0, 1}. Describe all functions T; V—> V. There are four altogether. Label them as 
T,, T,, T3, T, and make a multiplication table showing the composition of each pair. Indicate 
which functions are one-to-one on V and give their inverses. 

2. Let V = {0,1,2$. Describe all functions T: V— V for which T(V) = V. There are six 
altogether. Label them as 7,,..., 7, and make a multiplication table showing the com- 
position of each pair. Indicate which functions are one-to-one on V, and give their inverses. 


In each of Exercises 3 through 12, a function T: V,— V, is defined by the formula given for 
T(x, y), where (x, y) is an arbitrary point in V,. In each case determine whether T is one-to-one 
on V,. If it is, describe its range T(V.); for each point (u, v) in T(V,), let (x, y) = T~1(u, v) and 
give formulas for determining x and y in terms of u and v. 


3. T(x, y) = (y, x). 8. T(x, y) = (e”, e”). 

4. T(x, y) = (x, —y). 9. T(x, y) = (x, 1). 

5. T(x, y) = (x, 0). 10. T(x, y) =(x + 1,y +1). 
6. T(x, y) = (x, x). 11. Tix, y) = (* —y,x + y). 
7. T(x, y) = (x, y’). 12. T(x, y) = (2x —y,x + y). 
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In each of Exercises 13 through 20, a function T: V3— V3 is defined by the formula given for 
T(x, y, z), where (x, y, z) is an arbitrary point in V3. In each case, determine whether T is one-to- 
one on V3. If it is, describe its range T(V3); for each point (u,v, w) in T(V3), let (x, y, z) = 
T~1(u, v, w) and give formulas for determining x, y, and z in terms of u, v, and w. 


13. T(x, y, Zz) = (z, y, x) 17. T(x, y,z) =(* + 1,y +1,2z—-—1). 
14. T(x, y,z) = (x, y, 0). 18. Toy y, Zz) = +1, y +232 + 3): 
15. T(x, y, 2) = (x, 2y, 3z). 19. T(x, y,z) =(x,x +y,x +y +2). 
16. T(x, y,z) =(*%,y,x +y +2). 20. T(x, y,z)=(x +y,y +2z,x +2). 


21. Let T: V— V bea function which maps V into itself. Powers are defined inductively by the 
formulas T?=J, T" = TT" for n >1. Prove that the associative law for composition 
implies the law of exponents: T”7" = T™*” . If Tis invertible, prove that T” is also invertible 
and that (7”)1 = (T"1)". 


In Exercises 22 through 25, S and T denote functions with domain V and values in V. In 
general, ST # TS. If ST = TS, we say that S and T commute. 


22. If S and T commute, prove that (ST)" = S"7" for all integers n > 0. 

23. If Sand T are invertible, prove that ST is also invertible and that (ST)! = T-1S~1. In other 
words, the inverse of ST is the composition of inverses, taken in reverse order. 

24. If S and 7 are invertible and commute, prove that their inverses also commute. 

25. Let V be a linear space. If S and T commute, prove that 


(S +7)? =S*?*+2ST+T* = and (S + TT)? = 8S? + 38°7T + 3ST? + T?. 


Indicate how these formulas must be altered if ST ¥ TS. 

26. Let S and T be the linear transformations of V3 into V3 defined by the formulas S(x, y, z) = 
(z, y, x) and T(x, y,z) = (x,x +y,x + y +2), where (x, y, z) is an arbitrary point of V3. 
(a) Determine the image of (x, y, z) under each of the following transformations: ST, TS, 
ST — TS, S*, T*?, (ST)*, (TS)*, (ST — TS)*. 
(b) Prove that S and 7 are one-to-one on V3 and find the image of (u, v, w) under each of the 
following transformations: S71, T71, (ST)", (TS)1. 
(c) Find the image of (x, y, z) under (J — J)" for eachn > 1. 

27. Let V be the linear space of all real polynomials p(x). Let D denote the differentiation operator 
and let T denote the integration operator which maps each polynomial p onto the polynomial 
g given by g(x) = f 4 p(t) dt. Prove that DT = Ip but that TD # I,. Describe the null space 
and range of 7D. 

28. Let V be the linear space of all real polynomials p(x). Let D denote the differentiation operator 
and let T be the linear transformation that maps p(x) onto xp (x). 
(a) Let p(x) = 2 + 3x — x? + 4x° and determine the image of p under each of the following 
transformations: D, T, DT, TD, DT — TD, T?D* — D*®T?. 
(b) Determine those p in V for which T(p) = p. 
(c) Determine those p in V for which (DT — 2D)(p) = O. 
(d) Determine those p in V for which (DT — TD)"(p) = D"(p). 

29. Let V and D be as in Exercise 28 but let T be the linear transformation that maps p(x) onto 
xp(x). Prove that DT — TD = Iand that DT" — T"D =nT"" forn 22. 

30. Let Sand T be in Y(V, V) and assume that ST — TS =I. Provethat ST” — T"S =nT™ 
foralln >1. 

31. Let V be the linear space of all real polynomials p(x). Let R, S, T be the functions which map 
an arbitrary polynomial p(x) = cy + ox + °° + ¢,x" in V onto the polynomials r(x), s(x), 
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and ¢(x), respectively, where 


r(x) = p(0), s(x) = >) ae ea t(x) = > Ce, 
k=1 k=0 


(a) Let p(x) = 2 + 3x — x* + x? and determine the image of p under each of the following 
transformations: R, S, T, ST, TS, (TS)?, T2S?, S?T?, TRS, RST. 
(b) Prove that R, S, and 7 are linear and determine the null space and range of each. 
(c) Prove that T is one-to-one on V and determine its inverse. 
(d) Ifm > 1, express (7S)” and S"7” in terms of J and R. 

32. Refer to Exercise 28 of Section 2.4. Determine whether JT is one-to-one on V. If it is, describe 
its inverse. 


2.9 Linear transformations with prescribed values 


If V is finite-dimensional, we can always construct a linear transformation T: V—> W 
with prescribed values at the basis elements of V, as described in the next theorem. 


THEOREM 2.12. Let e,,...,€, be a basis for an n-dimensional linear space V. Let 
U,,...,U, ben arbitrary elements in a linear space W. Then there is one and only one linear 
transformation T: V — W such that 


(2.7) T(e,) =u, for k=1,2,...,n. 


This T maps an arbitrary element x in V as follows: 


(2.8) If x=>%x,e, then T(x) => x,u,. 
k=1 k=1 


Proof. Every x in V can be expressed uniquely as a linear combination of e,,...,e,; 
the multipliers x,,...,x, being the components of x relative to the ordered basis 
(€;,...,€,). If we define T by (2.8), it is a straightforward matter to verify that T is 
linear. If x = e, for some k, then all components of x are 0 except the kth, which is 1, so 
(2.8) gives T(e,) = u,, are required. 

To prove that there is only one linear transformation satisfying (2.7), let T’ be another 
and compute T’(x). We find that 


T(x) = T’ (2 = dT") = 3 eat = T(x). 


Since T’(x) = T(x) for all x in V, we have T’ = T, which completes the proof. 


EXAMPLE. Determine the linear transformation T: V,—> V, which maps the basis elements 
i = (1,0) andj = (0, 1) as follows: 


Ti)=i+j, Tij)=2i-j. 
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Solution. If x = x,i + xj is an arbitrary element of V,, then 7(x) is given by 


T(x) = xT) + x2T(j) = x + J) + X2(2i — J) = (1 + 2%2)i + (X, — x). 


2.10 Matrix representations of linear transformations 


Theorem 2.12 shows that a linear transformation T: V— W of a finite-dimensional 
linear space V is completely determined by its action on a given set of basis elements 
€;,...,e,. Now, suppose the space W is also finite-dimensional, say dim W = m, and let 
W1,...,W,, bea basis for W. (The dimensions nv and m may or may not be equal.) Since T 
has values in W, each element 7(e,,) can be expressed uniquely as a linear combination of the 
basis elements w,,..., Wm, Say 


T(e,) = Zz, tinWi » 


where f,,,..-, tm, are the components of T(e,,) relative to the ordered basis (w,,... 5 Wm): 
We shall display the m-tuple (4,,..., tmx) vertically, as follows: 


(2.9) 


t 


mk 


This array is called a column vector or a column matrix. We have such a column vector for 
each of the n elements T(e,),..., T(e,). We place them side by side and enclose them in 
one pair of brackets to obtain the following rectangular array: 


hy hie Lin 
toy ogg lon 
tnt tne ae Ss Linn! 


This array is called a matrix consisting of m rows and n columns. We callitanm by n matrix, 
or an m X n matrix. The first row is the 1 X nm matrix (t),, ty2,...,¢,,). The m x 1 
matrix displayed in (2.9) is the Ath column. The scalars ¢,, are indexed so the first subscript 
i indicates the row, and the second subscript k indicates the column in which t¢,, occurs. 
We call t,,, the ik-entry or the ik-element of the matrix. The more compact notation 


(tix), or (tip ya1> 


is also used to denote the matrix whose ik-entry is ¢,,. 
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Thus, every linear transformation T of an n-dimensional space V into an m-dimensional 
space W gives rise to an m X n matrix (f,,,) whose columns consist of the components of 
T(e;),..., T(e,) relative to the basis (w,,...,W,). We call this the matrix representation 
of T relative to the given choice of ordered bases (e,,...,e,) for Vand (w,,..., w,,) for 
W. Once we know the matrix (¢,,), the components of any element 7(x) relative to the 
basis (w,,..., W,,) can be determined as described in the next theorem. 


THEOREM 2.13. Let T be a linear transformation in £(V,W), where dim V =n and 
dimW=~m. Let(e,,...,e,) and (w,,..., W,) be ordered bases for V and W, respectively, 
and let (t,,,) be the m X n matrix whose entries are determined by the equations 


(2.10) TE)= > 105 for k=) 2.44450 
i=1 


Then an arbitrary element 


n 


(2.11) x => x,e, 
k=1 
in V with components (x,,...,X,) relative to (€,,..., @,) is mapped by T onto the element 
(2.12) T(x) = > YiW;i 
i=1 
in W with components (y,,...,5\Vm) relative to (Wy,...,Wm). The y, are related to the 


components of x by the linear equations 


(2.13) y=). for tH1, 2.4.65: 
k=l 


Proof. Applying T to each member of (2.11) and using (2.10), we obtain 


nr m 


T(x) = > x,T(e&) => xed taws = D (> for] w= Dy, 
k=1 k=1 i=l t=1 \k=1 1 


where each y, is given by (2.13). This completes the proof. 


Having chosen a pair of bases (e,,...,e,) and (w,,...,W,,) for Vand W, respectively, 
every linear transformation T: V—» W has a matrix representation (f,,). Conversely, if 
we start with any mn scalars arranged as a rectangular matrix (f,,) and choose a pair of 
ordered bases for V and W, then it is easy to prove that there is exactly one linear trans- 
formation 7: V > W having this matrix representation. We simply define 7 at the basis 
elements of V by the equations in (2.10). Then, by Theorem 2.12, there is one and only 
one linear transformation T: V > W with these prescribed values. The image 7(x) of an 
arbitrary point x in V is then given by Equations (2.12) and (2.13). 
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EXAMPLE |. Construction of a linear transformation from a given matrix. Suppose we 
Start with the 2 x 3 matrix 
3 1 —2 
1 oO 4] 


Choose the usual bases of unit coordinate vectors for V3 and V,. Then the given matrix 
represents a linear transformation T: V3—» V, which maps an arbitrary vector (x,, x2, x3) 
in V; onto the vector (y;, yz) in Vz according to the linear equations 


yy = 3x, + Neo 2Xe 
y2 — x1 + Ox. + 4x5. 


EXAMPLE 2. Construction of a matrix representation of a given linear transformation. 
Let V be the linear space of all real polynomials p(x) of degree < 3. This space has dimen- 
sion 4, and we choose the basis (1, x, x?, x°). Let D be the differentiation operator which 
maps each polynomial p(x) in V onto its derivative p’(x). We can regard D as a linear 
transformation of V into W, where W is the 3-dimensional space of all real polynomials 
of degree < 2. In W we choose the basis (1, x, x”). To find the matrix representation of D 
relative to this choice of bases, we transform (differentiate) each basis element of V and 
express it as a linear combination of the basis elements of W. Thus, we find that 


D(1) = 0 = 0+ Ox + 0x?, D(x) = 1 = 1 + Ox + 0x?, 
D(x?) = 2x = 0 + 2x + Ox?, D(x?) = 3x® = 0 + Ox + 3x?. 


The coefficients of these polynomials determine the columns of the matrix representation of 
D. Therefore, the required representation is given by the following 3 x 4 matrix: 


0 1 0 0 
0 0 2 0 
0 0 0 3 


To emphasize that the matrix representation depends not only on the basis elements but 
also on their order, let us reverse the order of the basis elements in W and use, instead, the 
ordered basis (x?, x, 1). Then the basis elements of V are transformed into the same poly- 
nomials obtained above, but the components of these polynomials relative to the new 
basis (x?, x, 1) appear in reversed order. Therefore, the matrix representation of D now 


becomes 
00 0 3 


00 2 0 
01 0 0 
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Let us compute a third matrix representation for D, usingthebasis(1,1 +x,1l+x+2, 
1+x-+ x* + x°) for V, and the basis (1, x, x) for W. The basis elements of V are trans- 
formed as follows: 


D(1) = 0, Di+x)=1, DAi+x+x)=1+4 2x, 
DO +x + x? + x3) = 1 + 2x + 3x?, 
so the matrix representation in this case is 


O11 1 
00 2 2 
00 0 3 


2.11 Construction of a matrix representation in diagonal form 


Since it is possible to obtain different matrix representations of a given linear transforma- 
tion by different choices of bases, it is natural to try to choose the bases so that the resulting 
matrix will have a particularly simple form. The next theorem shows that we can make 
all the entries 0 except possibly along the diagonal starting from the upper left-hand corner 
of the matrix. Along this diagonal there will be a string of ones followed by zeros, the 
number of ones being equal to the rank of the transformation. A matrix (t,,) with all 
entries ¢;, = 0 when i # k is said to be a diagonal matrix. 


THEOREM 2.14. Let V and W be finite-dimensional linear spaces, with dim V = n and 
dim W=m. Assume Te £(V, W) and let r = dim T(V) denote the rank of T. Then there 


exists a basis (€,,...,€,) for V and a basis (w,,... , Wm) for W such that 
(2.14) Tee) =w, for i=1,2,...,r, 

and 

(2.15) Tie =O for i=rt+l,...,n. 


Therefore, the matrix (t,,) of T relative to these bases has all entries zero except for the r 
diagonal entries 


hte = =, = 1. 


Proof. First we construct a basis for W. Since T(V) is a subspace of W with dim T(V) = 
r, the space T(V) has a basis of r elements in W, say w,,...,w,. By Theorem 1.7, these 
elements form a subset of some basis for W. Therefore we can adjoin elements w,,,,.. 
w,, SO that 


Laat 


(2.16) (Wy sacs Was Wedgscos-4 3. Woy) 


is a basis for W. 
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Now we construct a basis for V. Each of the first r elements w, in (2.16) is the image of at 
least one element in V. Choose one such element in V and call ite;. Then 7(e,) = w, for 
i=1,2,...,r so (2.14) is satisfied. Now let k be the dimension of the null space M(T). 
By Theorem 2.3 we haven =k +r. Since dim M(T) =k, the space N(T) has a basis 
consisting of k elements in V which we designate as e,,,,...,,,,- For each of these 
elements, Equation (2.15) is satisfied. Therefore, to complete the proof, we must show 
that the ordered set 


(2.17) (G15. be 9 Ory Cpita ees yep2%) 


is a basis for V. Since dim V =n =r +k, we need only show that these elements are 
independent. Suppose that some linear combination of them is zero, say 


T-+k 


(2.18) d ce, =O. 
i=1 


Applying T and using Equations (2.14) and (2.15), we find that 


r+k r 


> ¢;T(e,) = > C;W; = O ‘ 
i=1 i=l 


But w,,..., Ww, are independent, and hence c, = ++: =c,=0. Therefore, the first r 
terms in (2.18) are zero, so (2.18) reduces to 


r+k 
> C;e; = O . 
t=r+1 
But ¢€,.1,.--,5@,,, are independent since they form a basis for N(T), and hence c,,, = 


-++=c,,, =0. Therefore, all the c,; in (2.18) are zero, so the elements in (2.17) forma 
basis for V. This completes the proof. 


EXAMPLE. We refer to Example 2 of Section 2.10, where D is the differentiation operator 
which maps the space V of polynomials of degree < 3 into the space W of polynomials of 
degree <2. In this example, the range T(V) = W, so T has rank 3. Applying the method 
used to prove Theorem 2.14, we choose any basis for W, for example the basis (1, x, x?). 
A set of polynomials in V which map onto these elements is given by (x, $x”, 3x). We 
extend this set to get a basis for V by adjoining the constant polynomial 1, which ts a basis 
for the null space of D. Therefore, if we use the basis (x, $x, $x°, 1) for V and the basis 
(1, x, x") for W, the corresponding matrix representation for D has the diagonal form 


1 0 0 0 
0 1 0 0 
00 1 0 
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2.12 Exercises 


In all exercises involving the vector space V,,, the usual basis of unit coordinate vectors is to be 
chosen unless another basis is specifically mentioned. In exercises concerned with the matrix of 
a linear transformation T: V-—> W where V = W, we take the same basis in both V and W unless 
another choice Is indicated. 

1. Determine the matrix of each of the following linear transformations of V,, into V,: 
(a) the identity transformation, 
(b) the zero transformation, 
(c) multiplication by a fixed scalar c. 
2. Determine the matrix for each of the following projections. 
(a) T: V3—> V,, where 7(X,, Xo, X3) = (X1, Xo). 
(b) 7: Vz— V,, where T(x,, Xo, X3) = (Xo, X3). 
(c) T: Vi— Va, where T(x, Xo, X35 X%4,%5) = (Xo, Xg, X4). 
3. A linear transformation 7: V,—> V2 maps the basis vectors i and j as follows: 


T@ =i+ 7, Tj) = 2i —j. 


(a) Compute 7(3i — 4j) and T?(3i — 4j) in terms of i and j. 
(b) Determine the matrix of T and of T°. 
(c) Solve part (b) if the basis (i,j) is replaced by (e,, é,), where e) =i —J, ep = 39 +f. 
4. A linear transformation T: V,— V, is defined as follows: Each vector (x, y) is reflected in 
the y-axis and then doubled in length to yield T(x, y). Determine the matrix of T and of T°. 
5. Let T: V3—» Vz be a linear transformation such that 


T(k) = 2i + 37 + 5k, TUG +k) =i, TGit+jt+hk)=j—k. 


(a) Compute 7(i + 2j + 3k) and determine the nullity and rank of T. 
(b) Determine the matrix of T. 

6. For the linear transformation in Exercise 5, choose both bases to be (e,, @2, e3), where e, = 
(2, 3,5), eg = (1, 0,0), e, = (0, 1, —1), and determine the matrix of T relative to the new 
bases. 

7. A linear transformation T: V3—» V, maps the basis vectors as follows: T(i) = (0,0), TG) = 
(1,1), TK) =(1, -1). 

(a) Compute 7(4i — j + k) and determine the nullity and rank of T. 

(b) Determine the matrix of T. 

(c) Use the basis (i,j, kK) in Vz and the basis (w,, w.) in V,, where w, = (1, 1), w. = (, 2). 
Determine the matrix of T relative to these bases. 

(d) Find bases (e,, @,, ¢3) for V3 and (w,, wo) for V, relative to which the matrix of T will be 
in diagonal form. 

8. A linear transformation T: V,—» V3 maps the basis vectors as follows: T(i) = (1,9, 1), 
Tj) = (-1,9, 1). 

(a) Compute 7(2i — 3j) and determine the nullity and rank of T. 

(b) Determine the matrix of 7. 

(c) Find bases (e,, ¢,) for V. and (w,, w., ws) for V; relative to which the matrix of T will be 
in diagonal form. 

9. Solve Exercise 8 if T(i) = (1,0, 1) and 7(j) = (1, 1, 1). 

10. Let Vand W be linear spaces, each with dimension 2 and each with basis (¢,, ¢2). Let 7: V>W 
be a linear transformation such that T(e, + e,) = 3e, + 9e,, T(3e, + 2e9) = Te, + 232. 
(a) Compute T(e, — e,) and determine the nullity and rank of T. 
(b) Determine the matrix of T relative to the given basis. 
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(c) Use the basis (e,, é.) for V and find a new basis of the form (e, + ae,, 2e, + bey) for W, 
relative to which the matrix of T will be in diagonal form. 


In the linear space of all real-valued functions, each of the following sets is independent and 
spans a finite-dimensional subspace V. Use the given set as a basis for V and let D: V-> V be 
the differentiation operator. In each case, find the matrix of D and of D? relative to this choice 
of basis. 


11. (sin x, cos x). 15. (—cos x, sin x). 

IZ. lex ge") 16. (sin x, cos x, x sin x, x cos x). 
13. (1,1 +x,1 +x + e%). 17. (e7 sin x, e* cos x). 

14. (e, xe”). 18. (e?* sin 3x, e?* cos 3x). 


19. Choose the basis (1, x, x?, x*) in the linear space V of all real polynomials of degree <3. 
Let D denote the differentiation operator and let 7: V—» V be the linear transformation 
which maps p(x) onto xp’(x). Relative to the given basis, determine the matrix of each of the 
following transformations: (a) T; (b) DT; (c) TD; (d) TD — DT; (e) T*; (f) T?D? — D?T?. 

20. Refer to Exercise 19. Let W be the image of V under TD. Find bases for Vand for W relative 
to which the matrix of TD is in diagonal form. 


2.13 Linear spaces of matrices 


We have seen how matrices arise in a natural way as representations of linear trans- 
formations. Matrices can also be considered as objects existing in their own right, without 
necessarily being connected to linear transformations. As such, they form another class of 
mathematical objects on which algebraic operations can be defined. The connection 
with linear transformations serves as motivation for these definitions, but this connection 
will be ignored for the moment. 

Let m and n be two positive integers, and let /,, , be the set of all pairs of integers (i, /) 
such that] <i<cm,1<j<n. Any function A whose domain is /,, , is called an m x n 
matrix. The function value A(i, j) is called the ij-entry or ij-element of the matrix and will 
be denoted also by a;;. It is customary to display all the function values in a rectangular 
array consisting of m rows and n columns, as follows: 


Qj, Ayn *** Ain 


Gq, Ugg *** Agy 


Gm1 Gm2 °°" Amn 
The elements a,; may be arbitrary objects of any kind. Usually they will be real or complex 
numbers, but sometimes it 1s convenient to consider matrices whose elements are other 
objects, for example, functions. We also denote matrices by the more compact notation 


A = (4,521 or A = (a;;). 


If m =n, the matrix is said to be a square matrix. A 1 x n matrix is called a row matrix; 
an m X 1 matrix is called a column matrix. 
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Two functions are equal if and only if they have the same domain and take the same 
function value at each element in the domain. Since matrices are functions, two matrices 
A = (a,;) and B = (6,,;) are equal if and only if they have the same number of rows, the 
same number of columns, and equal entries a;; = b;; for each pair (i, /). 

Now we assume the entries are numbers (real or complex) and we define addition of 
matrices and multiplication by scalars by the same method used for any real- or complex- 
valued functions. 


DEFINITION. Jf A = (a,;) and B = (b,;) are two m X n matrices and if c is any scalar, 
we define matrices A + Band cA as follows: 


A+B= (a;; ah b;;), cA = (ca,;) . 


The sum is defined only when A and B have the same size. 


l 2 —3 5 0 l 
A= and B= : 
—] 0 4 1 —2 3 
then we have 


6 2 —2 2 4 —6 -—-5 90-1 
A+B= 2A = ; (-1l)B= : 
0 —2 7 —2 0 8 —l 2 23 
We define the zero matrix O to be the m x n matrix all of whose elements are 0. With 
these definitions, it is a straightforward exercise to verify that the collection of all m x n 
matrices is a linear space. We denote this linear space by M,,,,. If the entries are real 
numbers, the space M,, ,, iS areal linear space. If the entries are complex, M,, , 1S a complex 
linear space. It is also easy to prove that this space has dimension mn. In fact, a basis for 


Mn.» Consists of the mn matrices having one entry equal to | and all others equal to 0. 
For example, the six matrices 


100 010 001 000 00 0 00 0 
000! {00 0; {00 0] %4|1 00] {01 0] 10 0 1)’ 


form a basis for the set of all 2 x 3 matrices. 


EXAMPLE. If 


2.14 Isomorphism between linear transformations and matrices 


We return now to the connection between matrices and linear transformations. Let V 
and W be finite-dimensional linear spaces with dim V =n and dim W =m. Choose a 
basis (e,,..., ,) for Vand a basis (w,,...,W,,) for W. In this discussion, these bases are 
kept fixed. Let &(V, W) denote the linear space of all linear transformations of V into 
W. If Te L(V, W), let m(T) denote the matrix of T relative to the given bases. We recall 
that m(T) is defined as follows. 
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The image of each basis element e, is expressed as a linear combination of the basis 
elements in W: 


(2.19) T(e,) = >dtazw, for k=1,2,...,n. 
i=1 


The scalar multipliers ¢,, are the ik-entries of m(T). Thus, we have 
(2.20) mT) = (tarp: 


Equation (2.20) defines a new function m whose domain is &(V, W) and whose values 
are matricesin M,, ,. Sinceevery m X n matrix is the matrix m(T) for some Tin #(V, W), 
the range of mis M,, ,. The next theorem shows that the transformation m: 2(V, W)— 
Mn., is linear and one-to-one on #(V, W). 


THEOREM 2.15. ISOMORPHISM THEOREM. For all S and T in £&(V, W) and all scalars 
c, we have 
m(S + T)= m(S) + m(T) and m(cT) = cm(T). 
Moreover, 
m(S) = m(T) implies S=T, 


so m is one-to-one on £(V, W). 


Proof. The matrix m(T) is formed from the multipliers ¢,, in (2.19). Similarly, the 
matrix m(S) is formed from the multipliers s,, in the equations 


(2.21) S(e,)) => s,w, for k=1,2,...,n. 
i=1 
Since we have 


(S+ Te) = Slut tam and (€TV(G,) = ¥ (Clam, 


we obtainm(S + T) = (5; + t,,) = m(S) + m(T)and m(cT) = (ct) = cm(T). This proves 
that m is linear. 

To prove that m is one-to-one, suppose that m(S) = m(T), where S = (s,,) and T = 
(t;,). Equations (2.19) and (2.21) show that S(e,) = T(e,) for each basis element e,, 
so S(x) = T(x) for all x in V, and hence S = T. 


Note: The function m is called an isomorphism. For a given choice of bases, m 
establishes a one-to-one correspondence between the set of linear transformations 
L(V, W) and the set of m x n matrices M,,,,. The operations of addition and multipli- 
cation by scalars are preserved under this correspondence. The linear spaces #(V, W) 
and M,, , are said to be isomorphic. Incidentally, Theorem 2.11 shows that the domain 
of a one-to-one linear transformation has the same dimension as its range. Therefore, 
dim #(V, W) = dim M,,., = mn. 


If V = Wand if we choose the same basis in both V and W, then the matrix m(/) which 
corresponds to the identity transformation J: V > Vis ann X n diagonal matrix with each 
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diagonal entry equal to 1 and all others equal to 0. This is called the identity or unit matrix 
and is denoted by J or by /,. 


2.15 Multiplication of matrices 


Some linear transformations can be multiplied by means of composition. Now we shall 
define multiplication of matrices in such a way that the product of two matrices corresponds 
to the composition of the linear transformations they represent. 

We recall that if 7: U—> V and S: V — W are linear transformations, their composition 
ST: U — Wiis a linear transformation given by 


ST(x) = S[T(x)] for all x in U. 
Suppose that U, V, and W are finite-dimensional, say 
dim U =n, dim V = p, dimW=m. 
Choose bases for U, V, and W. Relative to these bases, the matrix m(S) is an m x p 
matrix, the matrix Tis a p X n matrix, and the matrix of ST is an m x n matrix. The 


following definition of matrix multiplication will enable us to deduce the relation m(ST) = 
m(S)m(T). This extends the isomorphism property to products. 


DEFINITION. Let A be any m xX p matrix, and let B be any p X n matrix, say 
A=(4,)¢j21 and B= (bjj)ijh1. 


Then the product AB is defined to be the m X n matrix C = (c¢,;) whose ij-entry is given by 


Dp 
(2.22) Ci5 = > Ai dy;- 


Note: The product AB is not defined unless the number of columns of A is equal to 
the number of rows of B. 


If we write A, for the ith row of A, and B’ for the jth column of B, and think of these as 
p-dimensional vectors, then the sum in (2.22) is simply the dot product A,- B’. In other 
words, the ij-entry of AB is the dot product of the ith row of A with the jth column of B: 


AB = (A, ° B’)73"). 
Thus, matrix multiplication can be regarded as a generalization of the dot product. 


6 
2 
sn 5 —1]. Since 4 is2 x 3and Bis3 x 2, 
0 2 


3 | 


EXAMPLE |. Let A = i: 4 
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the product AB is the 2 x 2 matrix 
AB = = : 
A, B} A, ‘ B 1 —7 
The entries of AB are computed as follows: 
A, B'=3-441°54+2-0=17, A,  B=3-64+1-(-1)+2:2=21, 


A, Bt=(—1)-441-54+0-051, A,-B?= (—1)-641+(—1)4+0-2 = —7. 


EXAMPLE 2. Let 


—2 

2 1 -—3 
A= and B= l 
1 2 4 ; 


Here A is 2 X 3 and Bis 3 x 1, so ABis the 2 x 1 matrix given by 


A,: B —9 
AB = = 
A,* B} 8 
since A,B} =2:(—2)+1°1+(—3):2 = —9and A,: BE =1-(—2)+2°:14+4-2 =8. 


EXAMPLE 3. If A and B are both square matrices of the same size, then both AB and BA 
are defined. For example, if 


1 2 3 4 
A= and B= : 
—1 1 ys -2 
13 8 —1 10 
AB = : = : 
2 —2 3 12 


This example shows that in general AB # BA. If AB = BA, we say A and B commute. 


we find that 


EXAMPLE 4. If J, is the p x p identity matrix, then J,4 = A for every p X n matrix A, 
and BI, = B for every m X p matrix B. For example, 
0 
1 2 3 
0; = ; 
4 5 6 
l 


Now we prove that the matrix of a composition ST is the product of the matrices m(S) 
and m(T). 


1 0 O}f2 0 


2 l 
1 2 3 

0 1 O;}]3] =43], 0 
4 5 6 

4 0 


0 0 14/4 
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THEOREM 2.16. Let T: U-— V and S: V ~ W be linear transformations, where U, V, W 
are finite-dimensional linear spaces. Then, for a fixed choice of bases, the matrices of S, T, 
and ST are related by the equation 


m(ST) = m(S)m(T). 


Proof. Assume dim U=n, dimV =p, dim W=~m. Let (u,,...,u,) bea basis for 
U, (v,,..., 0,) a basis for V, and (w,,..., w,,) a basis for W. Relative to these bases, we 
have 

m(S) = (s;;)7721, where S(v,) => 5,w, for k=1,2,...,p, 
i=1 
and 
p 
mT) = (t;)?21; where T(u;) = > tj, for j=1,2,...,n. 
k=1 


Therefore, we have 


Dp D m ™m Dp 
ST(u;) = S[T(u;)] = 2 tesS(0) me bie 2 s.Wi = 2 (sates Wi» 


so we find that 
m,n 
1,3 


m(ST) = (sats) "= m(S)m(T). 


We have already noted that matrix multiplication does not always satisfy the com- 
mutative law. The next theorem shows that it does satisfy the associative and distributive 
laws. 


THEOREM 2.17. ASSOCIATIVE AND DISTRIBUTIVE LAWS FOR MATRIX MULTIPLICATION. 
Given matrices A, B, C. 


(a) If the products A(BC) and (AB)C are meaningful, we have 
A(BC) = (AB)C (associative law). 
(b) Assume A and B are of the same size. If AC and BC are meaningful, we have 
(A + B)C = AC+ BC (right distributive law), 
whereas if CA and CB are meaningful, we have 
C(A + B) = CA + CB (left distributive law). 
Proof. These properties can be deduced directly from the definition of matrix multi- 
plication, but we prefer the following type of argument. Introduce finite-dimensional 
linear spaces U, V, W, X and linear transformations T: U>V, S:V->W, R:W>X 


such that, for a fixed choice of bases, we have 


A =m(R), B=m(S), C=m(T). 
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By Theorem 2.16, we have m(RS) = AB and m(ST) = BC. From the associative law for 
composition, we find that R(ST) = (RS)T. Applying Theorem 2.16 once more to this 
equation, we obtain m(R)m(ST) = m(RS)m(T) or A(BC) = (AB)C, which proves (a). The 
proof of (b) can be given by a similar type of argument. 


DEFINITION. If A is a square matrix, we define integral powers of A inductively as follows: 
A =I], A" = AA" for n>I1. 


2.16 Exercises 


1 2 2 2 
1 —4 2 
1IifA= zie B=|-1 3 |, C= |1 —1], compute B+ C, AB, 
=| 4 — 
5 —2 1 -3 


BA, AC, CA, A(2B — 3C). 
0 1 
2. Let A = se ole Find all 2 x 2 matrices B such that (a) AB =O; (b) BA =O. 


3. In each case find a, b, c, d to satisfy the given equation. 


0 0 1 Oj/a@ | 102 0 
10 0 0}/4 9 abcedaj\jo0 011 1 0 6 6 
(a) = © | -| i 
01 0 Offe 6 149 2})10 1 0 0 19 8 4 
00 0 I1)}\d 5 001 0 
4. Calculate AB — BA in each case. 
L222 411 
(a) A=]{2 1 2), B= |-4 2 O|; 
1 2 3 12 1 
2 oO 0 3 1 -2 
(b) A = 1 1 21, B=| 3 —2 4 
—1 yi 1 —3 5 Ill 


5. If A is a square matrix, prove that A"A™ = A™*” for all integers m >0,n >0. 


1 1 1 2 
6. Let A = | Verify that A? = and compute A”. 
0 1 0 1 


cos 9 —sin 0 cos 20 —sin 26 
7. Let A = . Verify that A? = and compute A”. 
sin@ cos @ sin20 cos 26 
1 1 1 l 2:3 
8. Let A =|]0 1 1]. Verify that 42 =|0 1 2]. Compute A®* and A*. Guess a general 


00 1 00 1 


formula for A” and prove it by induction. 
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1 0 
9. Let A = | . Prove that A? = 2A — IJ and compute A!”, 
—-1 1 


10. Find all 2 x 2 matrices A such that A? = O. 
11. (a) Prove that a2 x 2 matrix A commutes with every 2 x 2 matrix if and only if A commutes 
with each of the four matrices 


1 O 0 | 0 0 0 0 
oo; foo} {1 of [oa] 
(b) Find all such matrices A. 
12. The equation A? = J is satisfied by each of the 2 x 2 matrices 


Pa Pe 


where b and c are arbitrary real numbers. Find all 2 x 2 matrices A such that A? = J. 


2 —-1 7 6 
13. If A= | and B = | find 2 x 2 matrices C and D such that AC = B 
—2 3 9 8 


and DA = B. 
14. (a) Verify that the algebraic identities 


(A + B® =A? +2AB+ Band (A + B)(A — B) = A® — B? 


1 -1 1 0 
do not hold for the 2 x 2 matrices A = and B = : 
0 2 1 2 


(b) Amend the right-hand members of these identities to obtain formulas valid for all square 
matrices A and B. 
(c) For which matrices A and B are the identities valid as stated in (a)? 


2.17 Systems of linear equations 


Let A = (a,;) be a given m X n matrix of numbers, and let c,,...,c,, be m further 
numbers. A set of m equations of the form 


n 
(2.23) Sa for £1, 2,c2.,m, 
k=1 
is called a system of m linear equations in m unknowns. Here x,,...,%, are regarded as 
unknown. A solution of the system is any n-tuple of numbers (x,,..., x,) for which all the 


equations are satisfied. The matrix A is called the coefficient-matrix of the system. 
Linear systems can be studied with the help of linear transformations. Choose the usual 
bases of unit coordinate vectors in V, and in V,,. The coefficient-matrix A determines a 
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linear transformation, 7: V, — V,,, which maps an arbitrary vector x = (x,,...,x,)inV, 
onto the vector y = (y,.-.., Vm) in V,, given by the m linear equations 
n 
Ve=> aX; for P=), 2.582: 
k=1 
Let c = (qj, ..., Cm) be the vector in V,, whose components are the numbers appearing in 


system (2.23). This system can be written more simply as 
T(x) =. 


The system has a solution if and only if c is in the range of T. If exactly one x in V,, maps 
onto c, the system has exactly one solution. If more than one x maps onto c, the system 
has more than one solution. 


EXAMPLE 1. A system with no solution. The system x +y=1, x + y=2 has no 
solution. The sum of two numbers cannot be both 1 and 2. 


EXAMPLE 2. A system with exactly one solution. The system x + y=1,x— y=0O has 
exactly one solution: (x, y) = (4, 3). 


EXAMPLE 3. A system with more than one solution. The system x + y = 1, consisting 
of one equation in two unknowns, has more than one solution. Any two numbers whose 
sum is 1 gives a solution. 


With each linear system (2.23), we can associate another system 
n 
> 44%, = 0 for ia | aa ee! 1 
k=1 


obtained by replacing each c,; in (2.23) by 0. This is called the homogeneous system corre- 
sponding to (2.23). If ¢ # O, system (2.23) is called a nonhomogeneous system. A vector 
x in V,, will satisfy the homogeneous system if and only if 


T(x) = 0; 


where 7 is the linear transformation determined by the coefficient-matrix. The homogene- 
ous system always has one solution, namely x = O, but it may have others. The set of 
solutions of the homogeneous system is the null space of T. The next theorem describes the 
relation between solutions of the homogeneous system and those of the nonhomogeneous 
system. 


THEOREM 2.18. Assume the nonhomogeneous system (2.23) has a solution, say b. 

(a) If a vector x is a solution of the nonhomogeneous system, then the vector v = x — b 
is a solution of the corresponding homogeneous system. 

(b) [fa vector v is a solution of the homogeneous system, then the vector x =v + bisa 
solution of the nonhomogeneous system. 
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Proof. Let T: V,—V,, be the linear transformation determined by the coefficient- 
matrix, as described above. Since 6 is a solution of the nonhomogeneous system we have 
T(b) = c. Let x and v be two vectors in V,, such that v = x — b. Then we have 


T(v) = T(x — b) = T(x) — T(b) = T(x) — cc. 
Therefore 7(x) = c if and only if T(v) = O. This proves both (a) and (b). 


This theorem shows that the problem of finding all solutions of a nonhomogeneous 
system splits naturally into two parts: (1) Finding all solutions v of the homogeneous 
system, that is, determining the null space of T; and (2) finding one particular solution 5 of 
the nonhomogeneous system. By adding b to each vector v in the null space of 7, we thereby 
obtain all solutions x = v + 6b of the nonhomogeneous system. 

Let k denote the dimension of N(T) (the nullity of T). If we can find k independent 
solutions v,,..., 0, of the homogeneous system, they will form a basis for N(T), and we 
can obtain every v in N(T) by forming all possible linear combinations 


V= Vy Tit + LY, 
where f,,..., ¢,, are arbitrary scalars. This linear combination is called the general solution 


of the homogeneous system. If b is one particular solution of the nonhomogeneous system, 
then all solutions x are given by 


x=b+ tv, +--+ + t0,.- 


This linear combination is called the general solution of the nonhomogeneous system. 
Theorem 2.18 can now be restated as follows. 


THEOREM 2.19. Let T: V,,—> Vm be the linear transformation such that T(x) = y, where 
X= (X1,---5%Xn)s V = YWi>--+ > Vm) and 


n 
Ve= > ak, for ba"). 2 Wh 
k=1 


Let k denote the nullity of T. If v,,..., 0, are k independent solutions of the homogeneous 
system T(x) = O, and if b is one particular solution of the nonhomogeneous system T(x) = c, 
then the general solution of the nonhomogeneous system is 


x=b+t0,+°°°+ 4,, 
where t,,..., t, are arbitrary scalars. 
This theorem does not tell us how to decide if a nonhomogeneous system has a particular 
solution b, nor does it tell us how to determine solutions v,,..., v, of the homogeneous 


system. It does tell us what to expect when the nonhomogeneous system has a solution. 
The following example, although very simple, illustrates the theorem. 
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EXAMPLE. The system x + y = 2 has for its associated homogeneous system the equation 
x +y=0. Therefore, the null space consists of all vectors in V, of the form (t, —f), 
where ¢ is arbitrary. Since (t, —t) = ¢(1, —1), this is a one-dimensional subspace of V, 
with basis (1, —1). A particular solution of the nonhomogeneous system is (0, 2). There- 
fore the general solution of the nonhomogeneous system is given by 


(x, y) = (0, 2) + #1, —1) or ea yr? -f, 


where ¢ is arbitrary. 


2.18 Computation techniques 


We turn now to the problem of actually computing the solutions of a nonhomogeneous 
linear system. Although many methods have been developed for attacking this problem, 
all of them require considerable computation if the system is large. For example, to solve 
a system of ten equations in as many unknowns can require several hours of hand com- 
putation, even with the aid of a desk calculator. 

We shall discuss a widely-used method, known as the Gauss-Jordan elimination method, 
which is relatively simple and can be easily programmed for high-speed electronic computing 
machines. The method consists of applying three basic types of operations on the equations 
of a linear system: 

(1) Interchanging two equations; 

(2) Multiplying all the terms of an equation by a nonzero scalar; 

(3) Adding to one equation a multiple of another. 

Each time we perform one of these operations on the system we obtain a new system having 
exactly the same solutions. Two such systems are called equivalent. By performing these 
operations over and over again in a systematic fashion we finally arrive at an equivalent 
system which can be solved by inspection. 

We shall illustrate the method with some specific examples. It will then be clear how the 
method 1s to be applied in general. 


EXAMPLE 1. A system with a unique solution. Consider the system 


2x — Sy + 4z = —3 
x—2y+ z=5 
x — 4y + 6z = 10. 


This particular system has a unique solution, x = 124, y = 75, z = 31, which we shall 
obtain by the Gauss-Jordan elimination process. To save labor we do not bother to copy 
the letters x, y, z and the equals sign over and over again, but work instead with the aug- 
mented matrix 


(2.24) 1-2 1| 5 
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obtained by adjoining the right-hand members of the system to the coefficient matrix. The 
three basic types of operation mentioned above are performed on the rows of the augmented 
matrix and are called row operations. At any stage of the process we can put the letters 
x, y, z back again and insert equals signs along the vertical line to obtain equations. Our 
ultimate goal is to arrive at the augmented matrix 


1 0 0| 124 
(2.25) 010] 75 
001] 31 


after a succession of row operations. The corresponding system of equations is x = 124, 
y = 75, z = 31, which gives the desired solution. 

The first step is to obtain a | in the upper left-hand corner of the matrix. We can do this 
by interchanging the first row of the given matrix (2.24) with either the second or third 
row. Or, wecan multiply the first row by §. Interchanging the first and second rows, we get 


1 —2 1 5 


The next step is to make all the remaining entries in the first column equal to zero, leaving 
the first row intact. To do this we multiply the first row by —2 and add the result to the 
second row. Then we multiply the first row by —1 and add the result to the third row. 
After these two operations, we obtain 


(2.26) 0 -1 2] -13). 


—1 2 


—2 5 
adjacent to the two zeros. We can obtain a 1 in its upper left-hand corner by multiplying 
the second row of (2.26) by —1. This gives us the matrix 


—13 
Now we repeat the process on the smaller matrix | i which appears 


1 —2 1] 5 
0 1 —2 | 13]. 


Ea 
(2.27) O a Sol. 
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At this stage, the corresponding system of equations is given by 


x—-2y+ z= 5 
y—2z= 13 
Z = 31). 


These equations can be solved in succession, starting with the third one and working 
backwards, to give us 


z=31, y=13+22=134+62=75, x=54+2y—2=5+4150—31 = 124. 


Or, we can continue the Gauss—Jordan process by making all the entries zero above the 
diagonal elements in the second and third columns. Multiplying the second row of (2.27) 
by 2 and adding the result to the first row, we obtain 


0 —3 | 31 
0 1 —2] 13]. 
0 0 1} 31 


Finally, we multiply the third row by 3 and add the result to the first row, and then multiply 
the third row by 2 and add the result to the second row to get the matrix in (2.25). 


EXAMPLE 2. A system with more than one solution. Consider the following system of 3 
equations in 5 unknowns: 


2x—sy+t4z+ u-—v=—3 
(2.28) x—2y+ z- utv=5 
x — 4y + 6z + 2u—v=10. 


The corresponding augmented matrix 1s 


The coefficients of x, y, z and the right-hand members are the same as those in Example 1. 
If we perform the same row operations used in Example |, we finally arrive at the augmented 
matrix 


I 0 0 —16 19 | 124 
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The corresponding system of equations can be solved for x, y, and z in terms of u and v, 
giving us 
x = 124 + lou — 19v 


y= 754+ 9u— Ilv 
z= 31+ 3u— 4v. 


If we let u = ¢t, and v = f,, where ¢, and ¢, are arbitrary real numbers, and determine 
x, y, z by these equations, the vector (x, y, Zz, u, v) in V; given by 


(x, y, Z5 u, v) = (124 + 16t, p= 19f,, 75 + Ot, ==: l1t., 31 + 3t, eo 4t., bs te) 
is a solution. By separating the parts involving ¢, and t,, we can rewrite this as follows: 
(x, y, Z, u,v) = (124, 75, 31, 0, 0) + £,(16, 9, 3, 1,0) + 4,(—19, —11, —4,0, 1). 


This equation gives the general solution of the system. The vector (124, 75, 31, 0, 0) isa 
particular solution of the nonhomogeneous system (2.28). The two vectors (16, 9, 3, 1, 0) 
and (—19, —11, —4, 0, 1) are solutions of the corresponding homogeneous system. Since 
they are independent, they form a basis for the space of all solutions of the homogeneous 
system. 


EXAMPLE 3. A system with no solution. Consider the system 


2x — Sy + 4z = —3 
(2.29) x—-2y+ z= 5 
x—4y+5z= 10. 


This system is almost identical to that of Example 1 except that the coefficient of z in the 
third equation has been changed from 6 to 5. The corresponding augmented matrix is 


Applying the same row operations used in Example 1 to transform (2.24) into (2.27), we 
arrive at the augmented matrix 


t= 11s 
(2.30) 0 1 —2] 13). 
0 oO 0o|31 


When the bottom row is expressed as an equation, it states that 0 = 31. Therefore the 
original system has no solution since the two systems (2.29) and (2.30) are equivalent. 
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In each of the foregoing examples, the number of equations did not exceed the number 
of unknowns. If there are more equations than unknowns, the Gauss-Jordan process is 
still applicable. For example, suppose we consider the system of Example 1, which has the 
solution x = 124, y = 75,z = 31. If we adjoin a new equation to this system which is also 
satisfied by the same triple, for example, the equation 2x — 3y + z = 54, then the 
elimination process leads to the agumented matrix 


1 0 Of 124 
0 1 O} 75 
0 0 1] 31 
0 0 0 0 


with a row of zeros along the bottom. But if we adjoin a new equation which is not satisfied 
by the triple (124, 75, 31), for example the equation x + y + z = 1, then the elimination 
process leads to an augmented matrix of the form 


1 0 0} 124 
0 1 0| 75 
001] 31] 
00 0| a 


where a #0. The last row now gives a contradictory equation 0 = a which shows that 
the system has no solution. 


2.19 Inverses of square matrices 


Let A = (a,;) be a square n X n matrix. If there is another n X n matrix B such that 
BA =I, where Jis then X n identity matrix, then A is called nonsingular and B is called a 
left inverse of A. 

Choose the usual basis of unit coordinate vectors in V,, and let T: V,, — V,, be the linear 
transformation with matrix m(T) = A. Then we have the following. 


THEOREM 2.20. The matrix A is nonsingular if and only if T is invertible. If BA =I 
then B = m(T™). 


Proof. Assume that A is nonsingular and that BA = J. We shall prove that T(x) = O 
implies x = O. Given x such that 7(x) = O, let X be the n x 1 column matrix formed 
from the components of x. Since T(x) = O, the matrix product AX is ann X 1 column 
matrix consisting of zeros, so B(AX) is also acolumn matrix of zeros. But B(AX) = (BA)X = 
IX = X,so every component of x is 0. Therefore, Tis invertible, and the equation 77 = J 
implies that m(T)m(T™) = I or Am(T™) =/. Multiplying on the left by B, we find 
m(T-) = B. Conversely, if T is invertible, then 717 is the identity transformation so 
m(T-)m(7)is the identity matrix. Therefore A is nonsingular and m(7™)A = I. 
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All the properties of invertible linear transformations have their counterparts for non- 
singular matrices. In particular, left inverses (if they exist) are unique, and every left 
inverse is also a right inverse. In other words, if A is nonsingular and BA = /, then 
AB =I. Wecall B the inverse of A and denote it by A~’. The inverse A is also non- 
singular and its inverse is A. 

Now we show that the problem of actually determining the entries of the inverse of a 
nonsingular matrix is equivalent to solving n separate nonhomogeneous linear systems. 

Let A = (a;;) be nonsingular and let A~! = (5,,;) be its inverse. The entries of A and 
A” are related by the 1” equations 


(2.31) 2 dine = 0:;; 


where 6;; = lifi=j, and 6,, = Oif ij. For each fixed choice of j, we can regard this 
as a nonhomogeneous system of n linear equations in nm unknowns ),,;, b2;,...,5,;. Since 
A is nonsingular, each of these systems has a unique solution, the jth column of B. All 
these systems have the same coefficient-matrix A and differ only in their right members. 
For example, if A isa 3 x 3 matrix, there are 9 equations in (2.31) which can be expressed 
as 3 separate linear systems having the following augmented matrices: 


Qyy Ayg Ag | | Ay, Ay, ay3 | O Qi, Ay a3} 0 
Az, As, Aog | O}, Qo, Az, og | 1], Az, Qe, Qo3 | O|. 
Az, 3g sg | 0 A3, 432, 33 | O A3, M32, Q33 | | 


If we apply the Gauss-Jordan process, we arrive at the respective augmented matrices 


10 0/4, 1 0 0] de 10 O| ds 
0 1 01] da], 0 1 0] deol, O 1 O| desl. 
00 1] dy, 0 0 11] doo 0 0 11] dos 


In actual practice we exploit the fact that all three systems have the same coefficient-matrix 
and solve all three systems at once by working with the enlarged matrix 


M1, Ge a;|/1 0 0 


Q3, Az, a33|90 0 1 
The elimination process then leads to 


by, Oy dys 
be: Bee bog} . 
0 0 11] by, bse dos 
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The matrix on the right of the vertical line is the required inverse. The matrix on the left 
of the line is the 3 x 3 identity matrix. 

It is not necessary to know in advance whether A is nonsingular. If A is singular (not 
nonsingular), we can still apply the Gauss-Jordan method, but somewhere in the process 
one of the diagonal elements will become zero, and it will not be possible to transform A 
to the identity matrix. 

A system of n linear equations in » unknowns, say 


n 
> iX~p = Ci; 2 earn | a 
k=1 


can be written more simply as a matrix equation, 
AX =C, 


where A = (a,;) is the coefficient matrix, and X and C are column matrices, 


xy Cy 

Xo Co 
i= ‘ C= 

Xn, Cn 


If A is nonsingular there is a unique solution of the system given by X¥ = A“!C. 


2.20 Exercises 


Apply the Gauss-Jordan elimination process to each of the following systems. If a solution 
exists, determine the general solution. 


l x+y+3z= 5 3. 3x —2y+5z+ u=1 
2x -y+4z=11 x+ y-—3z+2u=2 
—y+ z= 3. 6x + y—4z4+3u=7. 
2.3x +2y+ z=1 6. hy = 32+: wes 
5x +3y +3z=2 2x-—yrt z-—lu=2 
xt y= 2=1, Tx +y —7z+3u =3. 

3. 3x +2y+ z=1 7. x + y+2z+ 3ut+ 4v=0 
5x + 3y +3z=2 2x +2y + 7z + llu + 14v =0 
Ix +4y +5z =3. 3x + 3y + 62 + 10u + 15v = 0. 

4,.3x+2y+ z=1 8. x -—2y+ z+2u= -2 
3x +3y +3z =2 2x +3y—- z-—Su= 9 
7x + 4y + 5z = 3 4x—-— yt z— u= 5 
x+ y— z=0. Sx —3y +2z+ u= 3. 


9. Prove that the system x + y + 2z =2, 2x —y + 3z =2, 5x —y + az = 6, has a unique 
solution if a # 8. Find all solutions when a = 8. 
10. (a) Determine all solutions of the system 
Sx + 2y —6z + 2u = —1 
= PP 2S CS HZ. 
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(b) Determine all solutions of the system 


Sx +2y —6z +2u = —1 
C= pr 2 w=] 2 
x+ yt Zz = 6. 


11. This exercise tells how to determine all nonsingular 2 x 2 matrices. Prove that 


ae a 
= (ad — bc)I. 
c a|| —c a 


is nonsingular if and only if ad — be ¥ 0, in which case its inverse is 
d 


1 d —b 
ad — bc —C a 


Determine the inverse of each of the matrices in Exercises 12 through 16. 


a 
Deduce that 
Cc 


123 4 
23 4 
012 3 
12.}/ 21 114. 15. 
0012 
-1 12 
0001 
12 2 0100 0 07 
13.12 -1 1 202 0 0 0 
030100 
- 2 2 16. 
001020 
=e 000301 
mre Seen “i ee 000020 
li a 6 


2.21 Miscellaneous exercises on matrices 


1. If a square matrix has a row of zeros or a column of zeros, prove that it is singular. 
2. For each of the following statements about n x n matrices, give a proof or exhibit a counter 
example. 
(a) If AB + BA = O, then A?B3 = BA’. 
(b) If A and Bare nonsingular, then A + B is nonsingular. 
(c) If A and B are nonsingular, then AB is nonsingular. 
(d) If A, B, and A + Bare nonsingular, then A — B is nonsingular. 
(e) If A? = O, then A — J is nonsingular. 
(f) If the product of k matrices A, --- Aj, is nonsingular, then each matrix A; is nonsingular. 
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1 2 6 0 
3. If A = , find a nonsingular matrix P such that PAP = 
5 4 0 -1 
a 


4. The matrix A = 
i 


L = = 
, where i? = —1,a =4(1 + ./5), and b =i(1 —- ./5), has the prop- 

b 
erty that A? = A. Describe completely all 2 x 2 matrices A with complex entries such that 
AA =A. 

5. If A? = A, prove that (A + J)* = 1 + (2* — 1)A. 

6. The special theory of relativity makes use of a set of equations of the form x’ = a(x — vf), 
y =y,z =2,t' =a(t — vx/c*). Here v represents the velocity of a moving object, c the 
speed of light, and a = c/,/c? — v?, where |v| <c. The linear transformation which maps the 
two-dimensional vector (x, f) onto (x’,?t’) is called a Lorentz transformation. Its matrix 
relative to the usual bases is denoted by L(v) and is given by 


1 —v 
L(v) = ‘| ; 
—vc* 1 


Note that L(v) is nonsingular and that L(O) = J. Prove that L(v)L(u) = L(w), where w = 
(u + v)c?/(uv + c*). In other words, the product of two Lorentz transformations is another 
Lorentz transformation. 

7. If we interchange the rows and columns of a rectangular matrix A, the new matrix so obtained 
is called the transpose of A and is denoted by A’. For example, if we have 


1 4 
E23 
a-| | then A’ =]2 5 
4 5 6 
3 6 
Prove that transposes have the following properties: 
(a) (A)' =A. (b) (A + B)' = At + B’. (c) (cA)! = cA’. 
(d) (AB)! = BtA!. (e) (A) = (A7)! if A is nonsingular. 
8. A square matrix A is called an orthogonal matrix if AA’ = J. Verify that the 2 x 2 matrix 


sin 0 cos 6 

that its rows, considered as vectors in V,,, form an orthonormal set. 
9. For each of the following statements about n x n matrices, give a proof or else exhibit a 

counter example. 

(a) If A and B are orthogonal, then A + B is orthogonal. 

(b) If A and B are orthogonal, then AB is orthogonal. 

(c) If A and B are orthogonal, then B is orthogonal. 
10. Hadamard matrices, named for Jacques Hadamard (1865-1963), are those n xn matrices 
with the following properties: 

I. Each entry is 1 or —1. 


cos9 —sin 6] | 
is orthogonal for each real 6. If A is any m x n orthogonal matrix, prove 


II. Each row, considered as a vector in V,,, has length a n. 

III. The dot product of any two distinct rows is 0. 

Hadamard matrices arise in certain problems in geometry and the theory of numbers, and 
they have been applied recently to the construction of optimum code words in space com- 
munication. In spite of their apparent simplicity, they present many unsolved problems. The 
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main unsolved problem at this time is to determine all x for which ann xX n Hadamard matrix 
exists. This exercise outlines a partial solution. 

(a) Determine all 2 x 2 Hadamard matrices (there are exactly 8). 

(b) This part of the exercise outlines a simple proof of the following theorem: If A isann x n 
Hadamard matrix, where n > 2, thenn is a multiple of 4. The proof is based on two very simple 
lemmas concerning vectors in n-space. Prove each of these lemmas and apply them to the 
rows of Hadamard matrix to prove the theorem. 


LEMMA |. If X, Y, Z are orthogonal vectors in V,,, then we have 


(X+ Y):(X+2Z)= |x|. 


LEMMA 2. Write X =(%1,..-,;%n), Y=Q15---,Vn)s Z=(%45---52n). If each 
component x;, y;, Z,; is either 1 or —1, then the product (x; + y,)(x; + 2,) is either 0 or 4. 


3 


DETERMINANTS 


3.1 Introduction 


In many applications of linear algebra to geometry and analysis the concept of a 
determinant plays an important part. This chapter studies the basic properties of determi- 
nants and some of their applications. 

Determinants of order two and three were intoduced in Volume I as a useful notation for 
expressing certain formulas in a compact form. We recall that a determinant of order two 
was defined by the formula 


M41 2 
(3.1) = 411422 — 12021 - 
Qo 22 
5 ils Sie ett : a2 : : : 
Despite similarity in notation, the determinant (written with vertical bars) is 
Q21 422 


a . | M1 Fie ; ; 
conceptually distinct from the matrix | (written with square brackets). The 
G21 Ae 


determinant is a number assigned to the matrix according to Formula (3.1). To emphasize 


this connection we also write 
Q1 2 
= det : 
Qo, Are 


Determinants of order three were defined in Volume I in terms of second-order determi- 
nants by the formula 


Qj, Aye 


Qo, Are 


Qy1 Ayo Ag 


Qs2 og Qo, Arg in Qo, Are 
ay3 


(3.2) det | @1 eg Gog | = Ay — Ape 


Q39 33 Q3; 33 Q3; Azo 


43, G32 33 


This chapter treats the more general case, the determinant of a square matrix of order” 
for any integer n > 1. Our point of view is to treat the determinant as a function which 
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assigns to each square matrix A a number called the determinant of A and denoted by det A. 
It is possible to define this function by an explicit formula generalizing (3.1) and (3.2). 
This formula is a sum containing n! products of entries of A. For large n the formula is 
unwieldy and is rarely used in practice. It seems preferable to study determinants from 
another point of view which emphasizes more clearly their essential properties. These 
properties, which are important in the applications, will be taken as axioms for a determi- 
nant function. Initially, our program will consist of three parts: (1) To motivate the 
choice of axioms. (2) To deduce further properties of determinants from the axioms. 
(3) To prove that there 1s one and only one function which satisfies the axioms. 


3.2 Motivation for the choice of axioms for a determinant function 


In Volume I we proved that the scalar triple product of three vectors A,, A,, Ag in 3-space 
can be expressed as the determinant of a matrix whose rows are the given vectors. Thus 
we have 

441 42 Ay 


A, X Ag: Az = det | de; deg aos] , 


Q3; 32 433 


where Ay = (411, G2, 413), Aa = (G21, Gez, G23), aNd Ag = (431, 52, Ags) - 

If the rows are linearly independent the scalar triple product is nonzero; the absolute 
value of the product is equal to the volume of the parallelepiped determined by the three 
vectors A,, Az, Ag. If the rows are dependent the scalar triple product is zero. In this case 
the vectors A,, A,, Aj are coplanar and the parallelepiped degenerates to a plane figure of 
zero volume. 

Some of the properties of the scalar triple product will serve as motivation for the choice 
of axioms for a determinant function in the higher-dimensional case. To state these 
properties in a form suitable for generalization, we consider the scalar triple product as a 
function of the three row-vectors A,, A,, A3. We denote this function by d; thus, 


d(A,, Az, As) = Ay X Ag: Az. 


We focus our attention on the following properties: 
(a) Homogeneity in each row. For example, homogeneity in the first row states that 


d(tA,, Ap, As) = t d(A,, Ag, As) ‘for every scalar f. 
(b) Additivity in each row. For example, additivity in the second row states that 
d(A,, Ag + C, Ag) = d(A,, Ag, Az) + d(A,, C, As) 
for every vector C. 
(c) The scalar triple product is zero if two of the rows are equal. 


(d) Normalization: 


d(i,j,k)=1, where i=(1,0,0), j=(0,1,0), k=(0,0,1). 
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Each of these properties can be easily verified from properties of the dot and cross 
product. Some of these properties are suggested by the geometric relation between the scalar 
triple product and the volume of the parallelepiped determined by the geometric vectors 
A,, Ay, A3;. The geometric meaning of the additive property (b) in a special case is of 
particular interest. If we take C = A, in (b) the second term on the right is zero because of 
(c), and relation (b) becomes 


(3.3) d(A,, A, + A, ’ As) = d(A, ) As, As) . 


This property is illustrated in Figure 3.1 which shows a parallelepiped determined by 
A,, Ag, Ag, and another parallelepiped determined by A,, A, + Ag, Ag. Equation (3.3) 
merely states that these two parallelepipeds have equal volumes. This is evident geometri- 
cally because the parallelepipeds have equal altitudes and bases of equal area. 


Volume = d(A,, A>, A3) Volume = d(A,, A, + Ag, A3) 


FIGURE 3.1 


Geometric interpretation of the property d(A,, Ag, As) = d(A,, Ay + Ag, Ag). The 
two parallelepipeds have equal volumes. 


3.3 A set of axioms for a determinant function 


The properties of the scalar triple product mentioned in the foregoing section can be 
suitably generalized and used as axioms for determinants of order n. If A = (a,;) is an 
n X n matrix with real or complex entries, we denote its rows by 4,,...,A,. Thus, the 
ith row of A is a vector in n-space given by 


A; => (a1; A325 o(e oy a,,,) « 


We regard the determinant as a function of then rows A,,..., A, and denote its values by 
d(A,,...,A,) or by det A. 


AXIOMATIC DEFINITION OF A DETERMINANT FUNCTION. A real- or complex-valued function 
d, defined for each ordered n-tuple of vectors A,,..., A, in n-space, is called a determinant 
function of order n if it satisfies the following axioms for all choices of vectors A,,..., Ay 
and C in n-space: 
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AXIOM I. HOMOGENEITY IN EACH ROW. [If the kth row A, is multiplied by a scalar t, 
then the determinant is also multiplied by t: 


Qe GTA psi Td ag Aisa a 
AXIOM 2. ADDITIVITY IN EACH ROW. For each k we have 
M Aye hgh FP Cyeans AZ) Sd Ay ees Ai) dA oC orn AD: 
AXIOM 3. THE DETERMINANT VANISHES IF ANY TWO ROWS ARE EQUAL: 
d(A,,...,A,) =0 if A;=A; for someiandj withi #/j. 
AXIOM 4. THE DETERMINANT OF THE IDENTITY MATRIX IS EQUAL TO |: 


AGG eed where I, is the kth unit coordinate vector. 


The first two axioms state that the determinant of a matrix is a linear function of each of 
its rows. This is often described by saying that the determinant is a multilinear function of 
its rows. By repeated application of linearity in the first row we can write 


p p 
d( 3 nC. Aas An} =) 1. d(Cus Ags uns Ay)s 
k=1 k=1 


where ¢,,...,¢, are scalars and C,,..., C, are any vectors in n-space. 
Sometimes a weaker version of Axiom 3 is used: 


AXIOM 3’. THE DETERMINANT VANISHES IF TWO ADJACENT ROWS ARE EQUAL: 


d(A,,...,A,) =0 if A, =Agsy  forsomek =1,2,...,n—1. 


It is a remarkable fact that for a given n there is one and only one function d which 
satisfies Axioms 1, 2, 3’ and 4. The proof of this fact, one of the principal results of this 
chapter, will be given later. The next theorem gives properties of determinants deduced 
from Axioms 1, 2, and 3’ alone. One of these properties is Axiom 3. It should be noted that 
Axiom 41s not used in the proof of this theorem. This observation will be useful later when 
we prove uniqueness of the determinant function. 


THEOREM 3.1. A determinant function satisfying Axioms 1, 2, and 3° has the following 
further properties: 
(a) The determinant vanishes if some row is O: 


WA) 253A.) = 0 if A,=O  forsomek. 
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(b) The determinant changes sign if two adjacent rows are interchanged: 
d(. ee 5 Ax, ya ee .) — —d(. se 5 Ansis Ax, ee ae 


(c) The determinant changes sign if any two rows A, and A, with i ¥ j are interchanged. 
(d) The determinant vanishes if any two rows are equal: 


d(A,,...,A,) =9 if A;=A; for somei andj with i # j. 
(e) The determinant vanishes if its rows are dependent. 


Proof. To prove (a) we simply take t = 0 in Axiom 1. To prove (b), let B be a matrix 
having the same rows as A except for row k and rowk + 1. Let both rows B, and B,,, 
be equal to A, + A,,,. Then det B = 0 by Axiom 3’. Thus we may write 


A(..., Ayn + Angi, Ay + Angi...) =O. 


Applying the additive property to row k and to rowk + | we can rewrite this equation as 
follows: 


| ee A one” Ry PRR or” Cay ey” A 
+ d(..., Apis, Apgi,s---) =O. 


The first and fourth terms are zero by Axiom 3’. Hence the second and third terms are 
negatives of each other, which proves (b). 

To prove (c) we can assume that i < 7. Wecan interchange rows A, and A, by performing 
an odd number of interchanges of adjacent rows. First we interchange row A, successively 
with the earlier adjacent rows A,_,, A;2,...,A,;. This requires j — i interchanges. 
Then we interchange row A, successively with the later adjacent rows A,.1, Aji2,--+ 5 Aj_t- 
This requires j — i — 1 further interchanges. Each interchange of adjacent rows reverses 
the sign of the determinant. Since there are (j — i) + (j — i — 1) = 2(j— i) — 1 inter- 
changes altogether (an odd number), the determinant changes sign an odd number of times, 
which proves (c). 

To prove (d), let B be the matrix obtained from A by interchanging rdws A; and 4;. 
Since A; = A; we have B = A and hence det B = det A. But by (c), det B = —det A. 
Therefore det A = 0. 

To prove (e) suppose scalars c,,...,c, exist, not all zero, such that >”_, c,4, = O. 
Then any row A, with c, ¥ 0 can be expressed as a linear combination of the other rows. 
For simplicity, suppose that A, is a linear combination of the others, say A, = >”, t,A,. 
By linearity of the first row we have 


d(A,, As, ee 6 9 A,,) = d( > teAy As, ee eg A, Pe d(A,,, As, ce 8g Ay): 
k=2 k=2 
But each term d(A,, A,,...,A,) in the last sum is zero since A, is equal to at least one of 
A,g,...,A,- Hence the whole sum is zero. If row A; is a linear combination of the other 


rows we argue the same way, using linearity in the ith row. This proves (e). 
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3.4 Computation of determinants 


At this stage it may be instructive to compute some determinants, using only the axioms 
and the properties in Theorem 3.1, assuming throughout that determinant functions exist. 
In each of the following examples we do not use Axiom 4 until the very last step in the 
computation. 


EXAMPLE 1. Determinant of a2 X 2 matrix. We shall prove that 


Qy9 


(3.4) det ia 


= 11429 — Qy2Qo) . 
Qo, aoe 


Write the row vectors as linear combinations of the unit coordinate vectors i = (1, 0) and 
j= (0,1): 

A, = (@y1, Ay.) = Qyi + ayof, Ag = (Gg, Ag) = Agi + Agoj. 
Using linearity in the first row we have 

d(A,, Ao) = d(ayyi + ay2j, Ao) = Ay, A(t, Az) + ay. d(J, Az). 

Now we use linearity in the second row to obtain 

A(i, Ay) = A(i, Agi + Ag2j) = Ag, A(i, i) + dog A(i, jf) = agg (i,j), 
since d(i, i) = 0. Similarly we find 

A(j, Ag) = A(j, Qqyt + Aggj) = ag, A(j, i) = —agq, d(i,j). 


Hence we obtain 
A(A,, Az) = (4,422 — 42421) A(t, J). 


But d(i,j) = 1 by Axiom 4, so d(A,, Ag) = @1dg, — Gy2Qeq,, aS asserted. 

This argument shows that if a determinant function exists for 2 x 2 matrices, then it 
must necessarily have the form (3.4). Conversely, it is easy to verify that this formula does, 
indeed, define a determinant function of order 2. Therefore we have shown that there is one 
and only one determinant function of order 2. 


EXAMPLE 2. Determinant of a diagonal matrix. A square matrix of the form 
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is called a diagonal matrix. Each entry a;; off the main diagonal (i ¥ /) is zero. We shall 
prove that the determinant of A is equal to the product of its diagonal elements, 


(3.5) det A = Q41do9°* * Ann. 


The kth row of A is simply a scalar multiple of the kth unit coordinate vector, A, = a,,J;. 
Applying the homogeneity property repeatedly to factor out the scalars one at a time we get 


det A = d(A.s ee ,A,) = d(a,,l,,.. . > Annln) = Q\1 °° ‘Ann AT,,.. . [,). 


an () 


This formula can be written in the form 
det A 7 Qi a ‘a,, det I, 


where J is the identity matrix. Axiom 4 tells us that det 7 = 1 so we obtain (3.5). 


EXAMPLE 3. Determinant of an upper triangular matrix. A square matrix of the form 


} Ms Uyn °°" Uy 
O Use *** Uan 
U= 
0 O u 


is called an upper triangular matrix. All the entries below the main diagonal are zero. We 
Shall prove that the determinant of such a matrix is equal to the product of its diagonal 
elements, 

det U = Uyyllog* * * Unn- 


First we prove that det U = 0 if some diagonal element u,, = 0. If the last diagonal 
element u,,,, 1s zero, then the last row is O and det U = 0 by Theorem 3.1(a). Suppose, then, 
that some earlier diagonal element u,,; is zero. To be specific, say u.. = 0. Then each of 
the n — | row-vectors U,,..., U, has its first two components zero. Hence these vectors 
span a subspace of dimension at most nm — 2. Therefore these n — 1 rows (and hence all 
the rows) are dependent. By Theorem 3.1(e), det U=0. In the same way we find that 
det U = Oif any diagonal element is zero. 

Now we treat the general case. First we write the first row U, as a sum of two row-vectors, 


U,=V,4+ Vi, 


where V; = [4,,0,...,O] and V, = [0, u,., u3,..., %,]. By linearity in the first row 
we have 


det U = det (V,, Uses e089 U,,) + det (Vi, Una e829 U,). 
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But det (V;, U,,..., U,,) = 0 since this is the determinant of an upper triangular matrix 
with a diagonal element equal to 0. Hence we have 


(3.6) det U = det (V,, U2,..., U,). 
Now we treat row-vector U, in a similar way, expressing it as a sum, 


U, ca Ve + V3 ) 
where 


V, = [0, Uso; 0, ay ecg 0] and V5 = [0, 0, Uo3 5 ee ey Usn| . 
We use this on the right of (3.6) and apply linearity in the second row to obtain 
(3.7) det U = det (V,, Vz, U3,...,U,), 


since det (V,, V,, Us,..., U,) =0. Repeating the argument on each of the succeeding 
rows in the right member of (3.7) we finally obtain 


det U = det (Vi, V2; eee yg Vi») > 


where (V,, V2,..., V,,) is a diagonal matrix with the same diagonal elements as U. There- 
fore, by Example 2 we have 


det U= UjyUo0° °° Unn> 
as required. 


EXAMPLE 4. Computation by the Gauss-Jordan process. The Gauss-Jordan elimination 
process for solving systems of linear equations is also one of the best methods for com- 
puting determinants. We recall that the method consists in applying three types of operations 
to the rows of a matrix: 

(1) Interchanging two rows. 

(2) Multiplying all the elements in a row by a nonzero scalar. 

(3) Adding to one row a scalar multiple of another. 

By performing these operations over and over again in a systematic fashion we can transform 
any square matrix A to an upper triangular matrix U whose determinant we now know 
how to compute. It is easy to determine the relation between det A and det U. Each time 
operation (1) is performed the determinant changes sign. Each time (2) is performed with 
a scalar c ¥ 0, the determinant is multiplied by c. Each time (3) is performed the deter- 
minant is unaltered. Therefore, if operation (1) is performed p times and if c,,..., c, are 
the nonzero scalar multipliers used in connection with operation (2), then we have 


(3.8) det A = (—1)(cyc,* ++ cg) det U. 


Again we note that this formula is a consequence of the first three axioms alone. Its proof 
does not depend on Axiom 4. 
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3.5 The uniqueness theorem 


In Example 3 of the foregoing section we showed that Axioms 1, 2, and 3 imply the 
formula det U = u4,Ug.° ++ u,, det 7. Combining this with (3.8) we see that foreveryn x n 
matrix A there is a scalar c (depending on A) such that 


(3.9) d(A,,...,A,) = edh,...51,): 


Moreover, this formula is a consequence of Axioms 1, 2, and 3 alone. From this we can 
easily prove that there cannot be more than one determinant function. 


THEOREM 3.2, UNIQUENESS THEOREM FOR DETERMINANTS. Let d be a function satisfying 
all four axioms for a determinant function of order n, and let f be another function satisfying 
Axioms 1, 2, and 3. Then for every choice of vectors A,,..., A,, in n-space we have 


(3.10) J(Ay,...,A,) = d(A,,...,4,)f(h,.--,1,)- 
In particular, if f also satisfies Axiom 4 we have f(A,,...,A,) = d(Ay,...,An)- 


Proof. Let g(A,,...,4,) =f(A1,...,A4,) —d(A,,...,4,)f(h,...,1,). We will 
prove that g(4,,...,A,) = 0 for every choice of A4,,...,A,. Since both d and / satisfy 
Axioms 1, 2, and 3 the same is true of g. Hence g also satisfies Equation (3.9) since this was 
deduced from the first three axioms alone. Therefore we can write 


(3.11) g(A,,...,4,)=ceth,...,4,), 


where c is a scalar depending on A. Taking A = Jin the definition of g and noting that d 
satisfies Axiom 4 we find 


(OR ae | nee 9 oes 0 ee eee 


Therefore Equation (3.11) becomes g(A4,,...,A,) =0. This completes the proof. 


3.6 Exercises 


In this set of exercises you may assume existence of a determinant function. Determinants of 
order 3 may be computed by Equation (3.2). 


1. Compute each of the following determinants. 


2 1 1 3 0 8 a 1 0 
(a) | 1 4 —4], (b) 5 0 Ts (c) |2 a 21 
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x y Zz 
2. If det}3 0 2] =1, compute the determinant of each of the following matrices: 
1 1 1 
2X: 2y- 22 x y Z x=—1l y=!l 2-1 
(a)|}3 O 1], (b) |3x +3 3y 3242], (c) 4 1 3 
1 1 1 ce eee ger l l l 


. (a) Prove that} a b c|=(b-—aj\(c —avc — Dd). 


a bh @- 


. Compute the determinant of each of the following matrices by transforming each of them to an 


upper triangular matrix. 


1 —1 1 1 1 1 1 1 1 1 1 1 
1 -—-1 -1 -1 a bead a bea 
a » (b ; Cc : 
(a) | 1-1 -1 (6) a hb? cc d? (c) a2 bh? ¢ qd? 
| 1 1 —1 a & 8 qd a’ bt c* dt 
| | l ] | | 
a 100 0) 
1 | 1 -—-1 -1 -1 
4a20 0 
| 1 -—-1 —1 | | 
d)|0 3 a3 O], e 
) ©) 1 -—-1 —-1l 1 —-1 l 
0 02a 4 
1 —1 1 -1 ] l 
0001 a 


1 -1 -!1 ] 1 —] 


. A lower triangular matrix A = (a,;) is a square matrix with all entries above the main diagonal 


equal to 0; that is, a;; = 0 whenever i <j. Prove that the determinant of such a matrix is 
equal to the product of its diagonal entries: det A = @,,d99° °° Qnn. 


. Let fi, fo, 21> 82 be four functions differentiable on an interval (a, 6). Define 


fi) f(x) 


F(x) = 
Ev(X)  £2(x) 


for each x in (a, b). Prove that 
f 1(X) f: 2(x) 
81(x) g5(x) 


fi@) f2@) 


F'(x) = 
£1 (%) £o(x) 


The product formula for determinants 8] 
7. State and prove a generalization of Exercise 6 for the determinant 
AiG) fe) fa) 


F(x) = |i) §2(x) &3(x) |. 
hy(x) hax) h(x) 


fi@) fo) 
A@® AO 


f, 1(X) f: 2(x) 
Sa (x) Vie (x) 


8. (a) If F(x) = , prove that F’(x) = 


(b) State and prove a corresponding result for 3 x 3 determinants, assuming the validity of 
Equation (3.2). 

9. Let Uand V be twon x n upper triangular matrices. 
(a) Prove that each of U + Vand UV is an upper triangular matrix. 
(b) Prove that det (UV) = (det U)(det V). 
(c) If det U 0 prove that there is an upper triangular matrix U~! such that UU"! = J, 
and deduce that det (U—!) = 1/det U. 

10. Calculate det A, det (A), and A™ for the following upper triangular matrix: 


nv Ww kL WN 


3 4 
2) 33 
0 2 
0 0 


oO Oo Oo WN 


3.7 The product formula for determinants 


In this section we use the uniqueness theorem to prove that the determinant of a product 
of two square matrices is equal to the product of their determinants, 


det (AB) = (det A)(det B), 


assuming that a determinant function exists. 
We recall that the product AB of two matrices A = (a,;) and B = (6,,) is the matrix 
C = (c,;) whose i, j entry is given by the formula 


(3.12) Cij = 2 Gabe 


The product is defined only if the number of columns of the left-hand factor A is equal to 
the number of rows of the right-hand factor B. This is always the case if both A and Bare 
Square matrices of the same size. 

The proof of the product formula will make use of a simple relation which holds between 
the rows of AB and the rows of A. We state this as a lemma. As usual, we let A; denote 
the ith row of matrix A. 
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LEMMA 3.3. Jf A isanm X n matrix and B ann X p matrix, then we have 
(AB); = A,B. 
That is, the ith row of the product AB is equal to the product of row matrix A, with B. 


Proof. Let B’ denote the jth column of Band let C = AB. Then the sum in (3.12) can 
be regarded as the dot product of the ith row of A with the jth column of B, 


C4 = A, B’. 
Therefore the ith row C; is the row matrix 
C, = [A;: B', A, B’,..., A; B®). 
But this is also the result of matrix multiplication of row matrix A; with B, since 
by bis eee Din | 
ALB 1Gis usage | = [A,- B',...,A,* B®]. 
Bri One °° Day 


Therefore C; = A,B, which proves the lemma. 


THEOREM 3.4. PRODUCT FORMULA FOR DETERMINANTS. For any n X n matrices A and B 
we have 
det (AB) = (det A)(det B). 
Proof. Since (AB); = A,B, we are to prove that 
d(A,B, eee 4 A,,B) — d(A,, eee 9 A,,) d(B,, oe 8 9 B,)- 


Using the lemma again we also have B; = (JB); = 1,B, where I is the n X n identity 
matrix. Therefore d(B,,..., B,) = dU,B,...,1,B), and we are to prove that 


d(A,B,...,A,B) = d(A,,...,A,)d(hB,...,1,B). 
We keep B fixed and introduce a function f defined by the formula 
f(A1,..+5An) = UA,B,..., A,B). 
The equation we wish to prove states that 


(3.13) f(A,, «2, Aq) = UA,,..+, Ag fl, . +5 %y)- 
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But now it is a simple matter to verify that f satisfies Axioms 1, 2, and 3 for a determinant 
function so, by the uniqueness theorem, Equation (3.13) holds for every matrix A. This 
completes the proof. 


Applications of the product formula are given in the next two sections. 


3.8 The determinant of the inverse of a nonsingular matrix 


We recall that a square matrix A is called nonsingular if it has a left inverse B such that 
BA =I. Ifaleft inverse exists it is unique and is also a right inverse, 4B = J. We denote 
the inverse by A~!. The relation between det A and det A™ is as natural as one could expect. 


THEOREM 3.5. If matrix A is nonsingular, then det A # 0 and we have 


1 


(3.14) det A = , 
det A 


Proof. From the product formula we have 
(det A)(det A~+) = det (AA“) = detJ=1. 
Hence det A # 0 and (3.14) holds. 


Theorem 3.5 shows that nonvanishing of det A is a necessary condition for A to be non- 
singular. Later we will prove that this condition is also sufficient. That is, if det A ~ 0 
then A7! exists. 


3.9 Determinants and independence of vectors 


A simple criterion for testing independence of vectors can be deduced from Theorem 3.5. 


THEOREM 3.6. A set of n vectors A,,...,A, in n-space is independent if and only if 
d(A,,...,A,) FO. 


Proof. We already proved in Theorem 3.2(e) that dependence implies d(A,,..., A,) = 
0. To prove the converse, we assume that A,,..., A, are independent and prove that 
d(A,,...,A,) #0. 

Let V,, denote the linear space of n-tuples of scalars. Since A,,..., A, aren independent 
elements in an n-dimensional space they form a basis for V,,._ By Theorem 2.12 there is a 
linear transformation T: V,,— V,, which maps these n vectors onto the unit coordinate 
vectors, 

T(A,) = I, fOr KS 1 2, ig ne 


Therefore there is ann X n matrix B such that 


A,B = I, fOr =A 2 ese: 
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But by Lemma 3.3 we have A,B = (AB),, where A is the matrix with rows A,,..., A,- 
Hence AB = I, so A is nonsingular and det A 4 0. 


3.10 The determinant of a block-diagonal matrix 
A O 

c= ; 
O B 


where A and B are square matrices and each O denotes a matrix of zeros, is called a block- 
diagonal matrix with two diagonal blocks A and B. Anexample is the 5 x 5 matrix 


A square matrix C of the form 


10000 
0 
C= 31. 
6 
K 9 


The diagonal blocks in this case are 


oO Oo oS & 
Co Oo OG = 
ay ft. YF © 
on NY OO © 


1 2 3 
1 O 
A= and B= 1|4 5 6}. 
0 1 
7 8 9 


The next theorem shows that the determinant of a block-diagonal matrix is equal to the 
product of the determinants of its diagonal blocks. 


THEOREM 3.7. For any two square matrices A and B we have 
A O 
(3.15) det = (det A)(det B). 
O B 


Proof. Assume A isn Xn and Bis m xX m. We note that the given block-diagonal 
matrix can be expressed as a product of the form 


A O A O|{I, O 

o B| |o 1,\|0 B 
where J, and /,, are identity matrices of orders n and m, respectively. Therefore, by the 
product formula for determinants we have 


A O A O I, O 
(3.16) det = det det ; 
O B O I, O B 
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A O 


Now we regard the determinant det | as a function of the m rows of A. This is 


m 


possible because of the block of zeros in the upper right-hand corner. It is easily verified 
that this function satisfies all four axioms for a determinant function of ordern. Therefore, 
by the uniqueness theorem, we must have 


A O 
det = det A. 
O I, 


I, O 
A similar argument shows that det b 4 = det B. Hence (3.16) implies (3.15). 


3.11 Exercises 


1. For each of the following statements about square matrices, give a proof or exhibit a counter 
example. 
(a) det (A + B) = det A + det B. 
(b) det {(A + B)*} = {det (A + B)P 
(c) det {(A + B)*} = det (A? + 2AB + B?) 
(d) det {(A + B)?} = det (A? + B?). 
2. (a) Extend Theorem 3.7 to block-diagonal matrices with three diagonal blocks: 


A OO 
det]O B O| = (det A)(det B)(det C). 
0OOC 


(b) State and prove a generalization for block-diagonal matrices with any number of diagonal 
blocks. 


1000 abed 
1 0 gh c d 
3. Let A = ; B= . Prove that det A = det and that 
abe 0 0 0 gih 
ef gh 0 1 
a 


b 
det B = det . 
ef 


4. State and prove a generalization of Exercise 3 for n x n matrices. 


a boo 
c dad 0 0 a b gh 

5. Let A = . Prove that det A = det det : 
ef gh c ad zZow 


x y Zw 
6. State and prove a generalization of Exercise 5 for n x n matrices of the form 


ee 


where B, C, D denote square matrices and O denotes a matrix of zeros. 
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7. Use Theorem 3.6 to determine whether the following sets of vectors are linearly dependent or 
independent. 
(a) A, = (1, —1,0), Ap = (0,1, —1), Ag = (2, 3, —1). 
(b) A, = (1, —1, 2,1), Ap = (—1, 2, —1, 0), Az = (3, —1, 1,0), Ay = (1, 0, 0, 1). 
(c) A; = (1,0, 0,0, 1), Ap = (1, 1,0, 0,0), 4g = (1,0, 1,0, 1), A, = 1, 1,0, 1, 1), 
A, = (0, 1,0, 1,0). 


3.12 Expansion formulas for determinants. Minors and cofactors 


We still have not shown that a determinant function actually exists, except in the 2 x 2 
case. In this section we exploit the linearity property and the uniqueness theorem to show 
that if determinants exist they can be computed by a formula which expresses every deter- 
minant of order 7 as a linear combination of determinants of order nm — 1. Equation (3.2) 
in Section 3.1 is an example of this formula in the 3 x 3 case. The general formula will 
suggest a method for proving existence of determinant functions by induction. 

Every row of ann X n matrix A can be expressed as a linear combination of the n unit 
coordinate vectors J,,...,/,. For example, the first row A, can be written as follows: 


Ay — > a,,l, , 
j=l 
Since determinants are linear in the first row we have 
G19). GAs Asses AD = d( Sais), Aas» An] = Says d(Ij, Aes +s An): 
j= j= 


Therefore to compute det A it suffices to compute d(/,, A,,..., A,) for each unit coordinate 
vector I;. 

Let us use the notation A,, to denote the matrix obtained from A by replacing the first 
row A, by the unit vector J;. For example, if m = 3 there are three such matrices: 


1 0 0 0 1 0 0 0 1 
Aj, = | 421 Gee Ao3], 12 = | 421 Gee Ags], Aig = | 4o1 ze gg 
Q31 Age ag Q3, 432 33 431 Azo Qgg 


Note that det 4,, = dU,;, A2,...,A,). Equation (3.17) can now be written in the form 
j=l 


This is called an expansion formula; it expresses the determinant of A asa linear combina- 
tion of the elements in its first row. 

The argument used to derive (3.18) can be applied to the kth row instead of the first row. 
The result is an expansion formula in terms of elements of the kth row. 
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THEOREM 3.8. EXPANSION BY COFACTORS. Let A,,; denote the matrix obtained from A 
by replacing the kth row A, by the unit coordinate vector I;. Then we have the expansion 
formula 


jal 


which expresses the determinant of A as a linear combination of the elements of the kth row. 
The number det A,, is called the cofactor of entry a,;. 


In the next theorem we shall prove that each cofactor is, except for a plus or minus sign, 
equal to a determinant of a matrix of order n — 1. These smaller matrices are called 
minors. 


DEFINITION. Given a square matrix A of order n > 2, the square matrix of order n — | 
obtained by deleting the kth row and the jth column of A is called the k, j minor of A and is 
denoted by A,,;. 


EXAMPLE. A matrix A = (a,,;) of order 3 has nine minors. Three of them are 
Qs ag Qo, Arg Qz, Ae 
Ay, = ) Ais = ) Ay = 7 
Az 33 Q3; 33 Az, Ago 
Equation (3.2) expresses the determinant of a 3 x 3 matrix as a linear combination of 
determinants of these three minors. The formula can be written as follows: 


det A = Qi det Ay = Aye det Axe + Qi3 det Ais . 
The next theorem extends this formula to the m X ncase for any n > 2. 
THEOREM 3.9. EXPANSION BY KTH-ROW MINORS. For any n Xn matrix A, n > 2, the 
cofactor of a,; is related to the minor A, by the formula 


Therefore the expansion of det A in terms of elements of the kth row is given by 


(3.21) det A = 3 (—1)*a,; det 4,5. 


J 


Proof. We illustrate the idea of the proof by considering first the special casek = j = 1. 
The matrix A), has the form 


1 0 0 0) 
Qo, Gee Aes Aon 
: Q31 Ago 33 Agn 
11™ 
Qni Qne ang Ann 
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By applying elementary row operations of type (3) we can make every entry below the | in 
the first column equal to zero, leaving all the remaining entries intact. For example, if we 
multiply the first row of A,, by —a, and add the result to the second row, the new second 
row becomes (0, @o2, dog, ..-, @2,). By a succession of such elementary row operations 
we obtain a new matrix which we shall denote by A®, and which has the form 


10 -:- QO 
O dy, *** Aan 
‘ O dg. *** azn 
Ay, = ° . 
0 Ano ~** Ann 


Since row operations of type (3) leave the determinant unchanged we have 
(3.22) det Af, = det Aj,. 
But A°, is a block-diagonal matrix so, by Theorem 3.7, we have 


det A?, = det A,,, 


where A,, is the 1, 1 minor of A, 


Qoo eee Asn 


A390 oeoe Asn 


Gng *""° ann 


Therefore det A|, = det A,,, which proves (3.20) fork =j=1. 
We consider next the special case k = 1, j arbitrary, and prove that 


(3.23) det Ai, = (—1)*? det A,,. 


Once we prove (3.23) the more general formula (3.20) follows at once because matrix A,, 
can be transformed to a matrix of the form Bi, by k — 1 successive interchanges of adjacent 
rows. The determinant changes sign at each interchange so we have 


where Bis ann X n matrix whose first row is J; and whose 1, j minor B,; is A,;. By (3.23), 
we have 

det Bi, = (—1)’* det B,; = (—1)’~ det A,;, 
so (3.24) gives us 


Therefore if we prove (3.23) we also prove (3.20). 
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We turn now to the proof of (3.23). The matrix A}, has the form 


0 1 0 
Ae) Ae; Aen 
; e 
Ai; = 
ani eoee Qnj eee ann 


By elementary row operations of type (3) we introduce a column of zeros below the | and 
transform this to 


0 0 1 0 0 
Qo, "°° * Ge 5-4 0 Qej41 “°° Gan 
0 _ ° 
Aj; 
Qni 77° Qn ,j-1 0 An,j+1 "Ann 


As before, the determinant is unchanged so det A°, = det A,,. The 1, / minor A,, has the 
form 


Qo, °° * Aggy Gojz1 °°° Gan 


Ay; — 


Ani eee Qnj-1 Qn j+1 oee Ann 


Now we regard det A}, as a function of the nm — | rows of Aj,;, say det A°, = f(A,,;). The 
function / satisfies the first three axioms for a determinant function of order nm — 1. There- 
fore, by the uniqueness theorem we can write 


(3.25) F(Ay;) =f) det Aj; , 


where J is the identity matrix of orderm — 1. Therefore, to prove (3.23) we must show that 
f(J) = (—1)"1. Now f(J) is by definition the determinant of the matrix 


@) Saar @) l @) ee @) 
1 00 0 0 
C=|0 +++ 100 <:+ 0] «jth row 
QO. 88:50 Oi. 4 wax 6 
0 oo. O @) @) cee 1 
T 


jth column 
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The entries along the sloping lines are all 1. The remaining entries not shown are all 0. 
By interchanging the first row of C successively with rows 2, 3,..., j/ we arrive at then x n 
identity matrix J after 7 — 1 interchanges. The determinant changes sign at each inter- 
change, so det C = (—1)’"1. Hence f(J) = (—1)*1, which proves (3.23) and hence (3.20). 


3.13 Existence of the determinant function 


In this section we use induction on 7, the size of a matrix, to prove that determinant 
functions of every order exist. Form = 2 we have already shown that a determinant function 
exists. We can also dispense with the case n = 1 by defining det [a,,] = a,. 

Assuming a determinant function exists of order n — 1, a logical candidate for a deter- 
minant function of order n would be one of the expansion formulas of Theorem 3.9, for 
example, the expansion in terms of the first-row minors. However, it is easier to verify the 
axioms if we use a different but analogous formula expressed in terms of the first-co/umn 
minors. 


THEOREM 3.10. Assume determinants of order n—1 exist. For any n Xn matrix 
A = (a;,), let f be the function defined by the formula 


(3.26) S(A, ee A,) — SY (-1* a, det Aj . 

j=l 
Then f satisfies all four axioms for a determinant function of order n. Therefore, by induction, 
determinants of order n exist for every n. 


Proof. We regard each term of the sum in (3.26) as a function of the rows of A and we 
write 


Si(Ay ge ee 9 A,) — (—1)'4a,, det Aj. 


If we verify that each /; satisfies Axioms | and 2 the same will be true for f. 

Consider the effect of multiplying the first row of A by a scalar ¢. The minor A,, is not 
affected since it does not involve the first row. The coefficient a,, is multiplied by ¢, so we 
have 


f,(tAy, 4, +. Ay) = tay, det Ay, = tf(A,, +--+, Ay): 


If 7 > 1 the first row of each minor A;, gets multiplied by ¢ and the coefficient a,, is not 
affected, so again we have 


F(tAi; As, 66s , A,,) — tf(A,, As, ae , A,,). 


Therefore each f; is homogeneous in the first row. 

If the kth row of A is multiplied by ¢, where k > 1, the minor A,, is not affected but a,, 1s 
multiplied by ¢, so f, is homogeneous in the kth row. If j #k, the coefficient a,, 1s not 
affected but some row of A,, gets multiplied by ¢. Hence every f; is homogeneous in the 
kth row. 
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A similar argument shows that each /; is additive in every row, so f satisfies Axioms | and 
2. We prove next that f satisfies Axiom 3’, the weak version of Axiom 3. From Theorem 
3.1, it then follows that f satisfies Axiom 3. 

To verify that f satisfies Axiom 3’, assume two adjacent rows of A are equal, say A, = 
A,,,. Then, except for minors A,; and A,,,,, each minor A,, has two equal rows so 
det A;, = 0. Therefore the sum in (3.26) consists only of the two terms corresponding to 
jJ=kandj=k +1, 


(3.27) (Ai, «++ An) = (—1)**"a,; det Ayr + (— 1)" ayy det Ansii . 


But Ayy = Agiia and ay; = G41, since A, = A,,,. Therefore the two terms in (3.27) 
differ only in sign, so f(A,,...,A,) = 0. Thus, f satisfies Axiom 3’. 

Finally, we verify that f satisfies Axiom 4. When A = I we have a,, = | and a,, = 0 for 
j > 1. Also, A,, is the identity matrix of ordern — 1, so each term in (3.26) is zero except 
the first, which is equal to 1. Hence f(/,,...,/,) = 1 sof satisfies Axiom 4. 


In the foregoing proof we could just as well have used a function f defined in terms of the 
kth-column minors A,, instead of the first-column minors A;,. In fact, if we let 


(3.28) f(Ay,..-, An) = (—1)" aj, det Aj, 


j=1 
exactly the same type of proof shows that this f satisfies all four axioms for a determinant 
function. Since determinant functions are unique, the expansion formulas in (3.28) and 
those in (3.21) are all equal to det A. 
The expansion formulas (3.28) not only establish the existence of determinant functions 


but also reveal a new aspect of the theory of determinants—a connection between row- 
properties and column-properties. This connection is discussed further in the next section. 


3.14 The determinant of a transpose 
Associated with each matrix A is another matrix called the transpose of A and denoted 


by A‘. The rows of A‘ are the columns of A. For example, if 


4 


1 
1 2 3 
A = ; then At=|2 5 
4 5 6 


A formal definition may be given as follows. 


DEFINITION OF TRANSPOSE. The transpose of anm Xn matrix A = (4,,)7%", is then X m 
matrix A‘ whose i, j entry is a;;. 


Although transposition can be applied to any rectangular matrix we shall be concerned 
primarily with square matrices. We prove next that transposition of a square matrix does 
not alter its determinant. 
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THEOREM 3.11. For any n X n matrix A we have 
det A = det A’. 


Proof. The proof is by induction on. Forn = 1 andn = 2 the result is easily verified. 
Assume, then, that the theorem is true for matrices of ordern — 1. Let A = (a,,) and let 
B = A‘ = (6,,). Expanding det A by its first-column minors and det B by its first-row 
minors we have 


det A = >(—1)'*"a,, det Aj,, det B= ¥(—1)**'b,, det B,;. 
Ss, 


j=1 


But from the definition of transpose we have b,; = a,, and B,; = (Aj). Since we are 
assuming the theorem is true for matrices of order n — 1 we have det B,,; = det Aj. 
Hence the foregoing sums are equal term by term, so det A = det B. 


3.15 The cofactor matrix 


Theorem 3.5 showed that if A is nonsingular then det A # 0. The next theorem proves 
the converse. That is, if det A # 0, then A~ exists. Moreover, it gives an explicit formula 
for expressing A~? in terms of a matrix formed from the cofactors of the entries of A. 

In Theorem 3.9 we proved that the i, 7 cofactor of a;; is equal to (—1)*+’ det A,;, where 
A,, 1s the i, j minor of A. Let us denote this cofactor by cof a,;. Thus, by definition, 


cof a;; = (—1)** det A,,. 


DEFINITION OF THE COFACTOR MATRIX. The matrix whose i, j entry is cof a;; is called the 
cofactor matrixt of A and is denoted by cof A. Thus, we have 


cof A = (cof a;;)Pj.1 = ((—1)"? det A;;)2 521. 


The next theorem shows that the product of A with the transpose of its cofactor matrix is, 
apart from a scalar factor, the identity matrix J. 


THEOREM 3.12. For any n X n matrix A with n > 2 we have 
(3.29) A(cof A)! = (det A)/. 
In particular, if det A # 0 the inverse of A exists and is given by 


A? = | (cof Ay. 
det A 


+ In much of the matrix literature the transpose of the cofactor matrix is called the adjugate of A. Some of 
the older literature calls it the adjoint of A. However, current nomenclature reserves the term adjoint for 


an entirely different object, discussed in Section 5.8. 
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Proof. Using Theorem 3.9 we express det A in terms of its kth-row cofactors by the 
formula 


(3.30) det A = } a,, cof a,;. 
j=1 


Keep k fixed and apply this relation to a new matrix B whose ith row is equal to the kth 
row of A for some i # k, and whose remaining rows are the same as those of A. Then 
det B = 0 because the ith and kth rows of B are equal. Expressing det B in terms of its 
ith-row cofactors we have 


j=1 
But since the ith row of B is equal to the Ath row of A we have 


Hence (3.31) states that 


n 


(3.32) > ay; cof a3; — 0 if k 7 l. 


j=1 


Equations (3.30) and (3.32) together can be written as follows: 


n 


det A if i=k 
(3.33) > ax; cof a;; = f if isk. 


j=1 


But the sum appearing on the left of (3.33) is the k, ientry of the product A(cof A)‘. There- 
fore (3.33) implies (3.29). 


As a direct corollary of Theorems 3.5 and 3.12 we have the following necessary and 


sufficient condition for a square matrix to be nonsingular. 


THEOREM 3.13. A square matrix A is nonsingular if and only if det A # 0. 


3.16 Cramer’s rule 


Theorem 3.12 can also be used to give explicit formulas for the solutions of a system of 
linear equations with a nonsingular coefficient matrix. The formulas are called Cramer’s 
rule, in honor of the Swiss mathematician Gabriel Cramer (1704-1752). 


THEOREM 3.14. CRAMER'S RULE. If a system of n linear equations in n unknowns 
x1 59° e © 9 Xno 
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has a nonsingular coefficient-matrix A = (a,;), then there is a unique solution for the system 
given by the formulas 


1 n 
(3.34) x; =—— /)b, cof a,,;, for j=1,2,...,n. 
det A 2 : 


Proof. The system can be written as a matrix equation, 


AX = B, 
Xx b, 
where X and B are column matrices, X = | °- |, B=]: |. Since A is nonsingular 
ae b, 
there is a unique solution X given by 
Z 1 
(3:35) X=A"'B= ay (cof A)'B. 


The formulas in (3.34) follow by equating components in (3.35). 


It should be noted that the formula for x, in (3.34) can be expressed as the quotient of two 
determinants, 


git det C; 
7 det A” 


where C; is the matrix obtained from A by replacing the jth column of A by the column 
matrix B. 


3.17 Exercises 


1. Determine the cofactor matrix of each of the following matrices: 


3 4 
2 —1 3 
1 2 P () Z ) 5 1 
a : b ‘ Cc ; 
(a) e A (b) ; 2p ek 
1 2 0 
—2 3 2 3 


2. Determine the inverse of each of the nonsingular matrices in Exercise 1. 
3. Find all values of the scalar 4 for which the matrix AJ — A is singular, if A is equal to 


0 2 11 =2 8 

0 3 
(a) 5 if? (b) |0 -1 —2], (c)} 19 —3 14 
2 2 -§ 2 -5 
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4. If Aisana xX nmatrix withn > 2, prove each of the following properties of its cofactor matrix: 
(a) cof (A‘) = (cof A). (b) (cof A)'A = (det A). 
(c) A(cof A)! = (cof A)'A (A commutes with the transpose of its cofactor matrix). 

5. Use Cramer’s rule to solve each of the following systems: 
(a) x +2y + 3z =8, 2x —y + 4z =7, =p oH, 
(b) x +y + 2z =0, IY Sy = 23, 2x + 5y + 3z =4, 

6. (a) Explain why each of the following is a Cartesian equation for a straight line in the xy-plane 
passing through two distinct points (x,, y,) and (Xg, y9). 


y 1 
set] 5 ft as det}x, y, 1] =0. 


Xg —Xy yo Vi Xe Ye 1 
2 J2 


(b) State and prove corresponding relations for a plane in 3-space passing through three 
distinct points. 
(c) State and prove corresponding relations for a circle in the xy-plane passing through three 
noncolinear points. 

7. Given n? functions f;;, each differentiable on an interval (a, 5), define F(x) = det [f;;(x)] for 
each x in (a, b). Prove that the derivative F’(x) is a sum of n determinants, 


F'(x) = >) det A;(x), 
i=1 


where A,(x) is the matrix obtained by differentiating the functions in the ith row of [f;;(x)]. 
8. Ann x n matrix of functions of the form W(x) = [u‘-))(x)], in which each row after the first 
is the derivative of the previous row, is called a Wronskian matrix in honor of the Polish mathe- 
matician J. M. H. Wronski (1778-1853). Prove that the derivative of the determinant of W(x) 
is the determinant of the matrix obtained by differentiating each entry in the last row of W(x). 


[Hint: Use Exercise 7.] 
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EIGENVALUES AND EIGENVECTORS 


4.1 Linear transformations with diagonal matrix representations 


Let T: V— V be a linear transformation on a finite-dimensional linear space V. Those 
properties of T which are independent of any coordinate system (basis) for V are called 
intrinsic properties of T. They are shared by all the matrix representations of T. If a basis 
can be chosen so that the resulting matrix has a particularly simple form it may be possible 
to detect some of the intrinsic properties directly from the matrix representation. 

Among the simplest types of matrices are the diagonal matrices. Therefore we might ask 
whether every linear transformation has a diagonal matrix representation. In Chapter 2 
we treated the problem of finding a diagonal matrix representation of a linear transfor- 
mation T: V > W, where dim V = n and dim W = m. In Theorem 2.14 we proved that 
there always exists a basis (e,,..., e,) for Vand a basis (w,,..., w,,) for W such that the 
matrix of 7 relative to this pair of bases is a diagonal matrix. In particular, if W = V 
the matrix will be a square diagonal matrix. The new feature now is that we want to use the 
same basis for both Vand W. With this restriction it is not always possible to find a diagonal 
matrix representation for 7. We turn, then, to the problem of determining which trans- 
formations do have a diagonal matrix representation. 


Notation: If A = (a,;) is a diagonal matrix, we write A = diag (@,,, dgo,..- 5 Qyn)> 


It is easy to give a necessary and sufficient condition for a linear transformation to have a 
diagonal matrix representation. 


THEOREM 4.1. Given a linear transformation T: V—> V, where dim V =n. If T has a 


diagonal matrix representation, then there exists an independent set of elements u,,...,U, 
in V and a corresponding set of scalars h,,..., 4, such that 

(4.1) 1@,)=Anu, for k= 1,2,...5,n. 

Conversely, if there is an independent set u,,...,U, in V and a corresponding set of scalars 


Ay» .+-5A, Satisfying (4.1), then the matrix 
A = diag (A,,...,4,) 


is a representation of T relative to the basis (u,,...,U,). 
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Proof. Assume first that T has a diagonal matrix representation A = (a,,) relative to 
some basis (e,,...,@,). The action of T on the basis elements is given by the formula 


n 
T(é,) = 2, Fei = Ayes 
i= 


since a,;, = 0 fori #k. This proves (4.1) with u, = e, and A, = a,,. 

Now suppose independent elements u,,...,u, and scalars /,,..., A, exist satisfying 
(4.1). Since u,,..., u, are independent they form a basis for V. If we define a,, = A, and 
a,, = 0 for i#k, then the matrix A = (a,,) is a diagonal matrix which represents 7 
relative to the basis (u,,...,U,). 


Thus the problem of finding a diagonal matrix representation of a linear transformation 
has been transformed to another problem, that of finding independent elements u,,..., u,, 
and scalars A,,..., A, to satisfy (4.1). Elements uw, and scalars 4,, satisfying (4.1) are called 
eigenvectors and eigenvalues of T, respectively. In the next section we study eigenvectors 
and eigenvaluesf in a more general setting. 


4.2 Eigenvectors and eigenvalues of a linear transformation 


In this discussion V denotes a linear space and S denotes a subspace of V. The spaces S 
and V are not required to be finite dimensional. 


DEFINITION. Let T: S—> V be a linear transformation of S into V. A scalar A is called an 
eigenvalue of T if there is a nonzero element x in S such that 


(4.2) LX) Ae. 


The element x is called an eigenvector of T belonging to 4. The scalar 4 is called an eigenvalue 
corresponding to x. 


There is exactly one eigenvalue corresponding to a given eigenvector x. In fact, if we 
have 7(x) = Ax and 7(x) = mux for some x # O, then Ax = ux sos= um. 


Note: Although Equation (4.2) always holds for x = O and any scalar 4, the definition 
excludes O as an eigenvector. One reason for this prejudice against O is to have exactly one 
eigenvalue 4 associated with a given eigenvector x. 


The following examples illustrate the meaning of these concepts. 


EXAMPLE 1. Multiplication by a fixed scalar. Let T: S—> V be the linear transformation 
defined by the equation 7(x) = cx for each x in S, where c is a fixed scalar. In this example 
every nonzero element of S is an eigenvector belonging to the scalar c. 


+ The words eigenvector and eigenvalue are partial translations of the German words Eigenvektor and 
Eigenwert, respectively. Some authors use the terms characteristic vector, or proper vector as synonyms for 
eigenvector. Eigenvalues are also called characteristic values, proper values, or latent roots. 
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EXAMPLE 2. The eigenspace E(A) consisting of all x such that T(x) = Ax. Let T: S>V 
be a linear transformation having an eigenvalue 4. Let E(A) be the set of all elements x in 
S such that T(x) = Ax. This set contains the zero element O and all eigenvectors belonging 
to A. It is easy to prove that E(A) is a subspace of S, because if x and y are in E(A) we have 


T(ax + by) = aT(x) + bT(y) = adx + bly = XMax + by) 


for all scalars a and b. Hence (ax + by) € E(A) so E(A) is a subspace. The space E(A) is 
called the eigenspace corresponding to A. It may be finite- or infinite-dimensional. If E(A) 
is finite-dimensional then dim E(A) > 1, since E(A) contains at least one nonzero element x 
corresponding to A. 


EXAMPLE 3. Existence of zero eigenvalues. If an eigenvector exists it cannot be zero, by 
definition. However, the zero scalar can be an eigenvalue. In fact, if 0 is an eigenvalue for 
x then T(x) = 0x = O, so x is in the null space of JT. Conversely, if the null space of T 
contains any nonzero elements then each of these is an eigenvector with eigenvalue 0. In 
general, E(A) is the null space of T — AT. 


EXAMPLE 4. Reflection in the xy-plane. Let S = V = V,(R) and let T be a reflection in 
the xy-plane. That is, let T act on the basis vectors i, j, k as follows: T(i) =i, T(j) =/J, 
T(k) = —k. Every nonzero vector in the xy-plane is an eigenvector with eigenvalue 1. 
The remaining eigenvectors are those of the form ck, where c #0; each of them has 
eigenvalue —1. 


EXAMPLE 5. Rotation of the plane through a fixed angle «. This example is of special interest 
because it shows that the existence of eigenvectors may depend on the underlying field of 
scalars. The plane can be regarded as a linear space in two different ways: (1) As a 2- 
dimensional real linear space, V = V,(R), with two basis elements (1, 0) and (0, 1), and 
with real numbers as scalars; or (2) as a 1-dimensional complex linear space, V = V,(C), 
with one basis element 1, and complex numbers as scalars. 

Consider the second interpretation first. Each element z ¥ 0 of V,(C) can be expressed 
in polar form, z = re’®. If T rotates z through an angle « then T(z) = re(@+#) = e@#z, 
Thus, each z ¥ 0 is an eigenvector with eigenvalue A = e’*. Note that the eigenvalue e’* is 
not real unless « is an integer multiple of 7. 

Now consider the plane as a real linear space, V,(R). Since the scalars of V,(R) are real 
numbers the rotation 7 has real eigenvalues only if « is an integer multiple of 7. In other 
words, if « is not an integer multiple of 7 then T has no real eigenvalues and hence no 
eigenvectors. Thus the existence of eigenvectors and eigenvalues may depend on the choice 
of scalars for V. 


EXAMPLE 6. The differentiation operator. Let V be the linear space of all real functions f 
having derivatives of every order on a given open interval. Let D be the linear transfor- 
mation which maps each f onto its derivative, D(/) =f’. The eigenvectors of D are those 
nonzero functions / satisfying an equation of the form 
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fi=if 


for some real A. This is a first order linear differential equation. All its solutions are given 
by the formula 


f(x) = ce, 
where c is an arbitrary real constant. Therefore the eigenvectors of D are all exponential 


functions f(x) = ce** with c #0. The eigenvalue corresponding to f(x) = ce** is A. In 
examples like this one where V is a function space the eigenvectors are called eigenfunctions. 


EXAMPLE 7. The integration operator. Let V be the linear space of all real functions 
continuous on a finite interval [a,b]. If fe V define g = T(/) to be that function given by 


ax)=|s(dt if a<x<b. 


The eigenfunctions of T (if any exist) are those nonzero f satisfying an equation of the form 


(4.3) [? f@ at = 4f@) 


for some real A. If an eigenfunction exists we may differentiate this equation to obtain the 
relation f(x) = Af’(x), from which we find f(x) = ce*’, provided A # 0. In other words, 
the only candidates for eigenfunctions are those exponential functions of the form f(x) = 
cet4 with c # Oand 2 #0. However, if we put x = a in (4.3) we obtain 


0 = Af(a) = Ace”. 


Since e*/4 is never zero we see that the equation T(f) = Af cannot be satisfied with a non- 
zero f, so T has no eigenfunctions and no eigenvalues. 


EXAMPLE 8. The subspace spanned by an eigenvector. Let T: S— V be a linear trans- 
formation having an eigenvalue A. Let x be an eigenvector belonging to A and let L(x) be 
the subspace spanned by x. That is, L(x) is the set of all scalar multiples of x. It is easy to 
show that T maps L(x) into itself. In fact, if y = cx we have 


T(y) = T(cx) = cT(x) = c(Ax) = A(cx) = dy. 


If c A Othen y # O so every nonzero element y of L(x) is also an eigenvector belonging 
to A. 


A subspace U of Sis called invariant under T if T maps each element of U onto an element 
of U. We have just shown that the subspace spanned by an eigenvector is invariant under T. 
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4.3 Linear independence of eigenvectors corresponding to distinct eigenvalues 


One of the most important properties of eigenvalues is described in the next theorem. As 
before, S denotes a subspace of a linear space V. 


THEOREM 4.2. Let u,,...,U, be eigenvectors of a linear transformation T: S—» V, and 
assume that the corresponding eigenvalues 4,,..., A, are distinct. Then the eigenvectors 
U,,..., U, are independent. 

Proof. The proof is by induction on k. The result is trivial when k = 1. Assume, then, 


that it has been proved for every set of k — 1 eigenvectors. Let u,,...,u, be k eigen- 
vectors belonging to distinct eigenvalues, and assume that scalars c, exist such that 


k 
(4.4) > cu; =O. 
Applying 7 to both members of (4.4) and using the fact that T(u;) = A,u,; we find 
k 
(4.5) > ¢Au, =O. 
t=1 
Multiplying (4.4) by A, and subtracting from (4.5) we obtain the equation 


k-1 
Dd cA; — 4,)u; = O. 
i=1 


But since u,,... , u,_, are independent we must have c,(A; — A,) = Oforeachi = 1,2,..., 
k — 1. Since the eigenvalues are distinct we have A, # A, fori# k soc, =O fori = I, 2, 

,k — 1. From (4.4) we see that c, is also 0 so the eigenvectors u,,..., u, are inde: 
pendent. 


Note that Theorem 4.2 would not be true if the zero element were allowed to be an eigen- 
vector. This is another reason for excluding O as an eigenvector. 


Warning: The converse of Theorem 4.2 does not hold. That is, if T has independent 
eigenvectors u,,..., u,, then the corresponding eigenvalues 4,,..., 4, need not be dis- 
tinct. For example, if T is the identity transformation, T(x) = x for all x, then every 
x # O is an eigenvector but there is only one eigenvalue, 4 = 1. 


Theorem 4.2. has important consequences for the finite-dimensional case. 


THEOREM 4.3. If dim V =n, every linear transformation T: V — V has at most n distinct 
eigenvalues. If T has exactly n distinct eigenvalues, then the corresponding eigenvectors form 
a basis for V and the matrix of T relative to this basis is a diagonal matrix with the eigenvalues 
as diagonal entries. 


Proof. If there were n + 1 distinct eigenvalues then, by Theorem 4.2, V would contain 
n+ 1 independent elements. This is not possible since dim V =n. The second assertion 
follows from Theorems 4.1 and 4.2. 


4.4 
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Note: Theorem 4.3 tells us that the existence of n distinct eigenvalues is a sufficient 
condition for T to have a diagonal matrix representation. This condition is not necessary. 
There exist linear transformations with less than n distinct eigenvalues that can be 
represented by diagonal matrices. The identity transformation is an example. All its eigen- 
values are equal to | but it can be represented by the identity matrix. Theorem 4.1 
tells us that the existence of m independent eigenvectors is necessary and sufficient for 
T to have a diagonal matrix representation. 


Exercises 


. (a) If T has an eigenvalue 4, prove that aT has the eigenvalue aA. 


(b) If x is an eigenvector for both T, and 7,, prove that x is also an eigenvector for aT, + bT,. 
How are the eigenvalues related ? 


. Assume T: V-> Vhas an eigenvector x belonging to an eigenvalue 4. Prove that xis an eigen- 


vector of T? belonging to A” and, more generally, x is an eigenvector of T” belonging to 4”. 
Then use the result of Exercise 1 to show that if P is a polynomial, then x is an eigenvector of 
P(T) belonging to P(A). 


. Consider the plane as a real linear space, V = V,(R) , and let T be a rotation of V through an 


angle of 7/2 radians. Although T has no eigenvectors, prove that every nonzero vector is an 
eigenvector for T?. 


. If T: V— V has the property that T? has a nonnegative eigenvalue /?, prove that at least one 


of A or —A is an eigenvalue for 7. [Hint: T? — PI = (T + A(T — AD) .] 


. Let V be the linear space of all real functions differentiable on (0, 1). If f E¢ V, define g = 


T(f) to mean that g(t) = of ‘(t) for all tin (0, 1). Prove that every real Ais an eigenvalue for 
T, and determine the eigenfunctions corresponding to 4. 


. Let V be the linear space of all real polynomials p(x) of degree <n. If pe V, define g = 


T(p) to mean that g(t) = p(t + 1) forall real t. Prove that T has only the eigenvalue 1. What 
are the eigenfunctions belonging to this eigenvalue ? 


. Let V be the linear space of all functions continuous on (— 0, +0) and such that the integral 


fr 0 f(t) at exists for all real x. If fe V, let g = T(f) be defined by the equation g(x) = 
ft f(t) dt. Prove that every positive 4 is an eigenvalue for T and determine the eigenfunctions 
corresponding to 4. 


. Let V be the linear space of all functions continuous on (— ©, + 0) and such that the integral 


fr. t f(t) dt exists for all real x. If fe V let g = T(/) be defined by the equation g(x) = 
fu t f(t) dt. Prove that every negative 4 is aneigenvalue for T and determine the eigenfunc- 
tions corresponding to A. 


. Let V = C(O, 7) be the real linear space of all real functions continuous on the interval [0, 7]. 


Let S be the subspace of all functions f which have a continuous second derivative in linear 
and which also satisfy the boundary conditions f(0) = f(7) =0. Let 7: S — V be the linear 
transformation which maps each f onto its second derivative, T(f) =f". Prove that the 
eigenvalues of T are the numbers of the form —n’, where n = 1, 2,..., and that the eigen- 
functions corresponding to —n? are f(t) = c, sin nt, where c, 4 0. 


. Let V be the linear space of all real convergent sequences {x,}. Define 7: V — V as follows: 


If x = {x,} is a convergent sequence with limit a, let T(x) = {y,}, where y, = a@ — x, for 
n >1. Prove that T has only two eigenvalues, 4 = 0 and A = —1, and determine the eigen- 
vectors belonging to each such 4. 


. Assume that a linear transformation T has two eigenvectors x and y belonging to distinct 


eigenvalues 4 and w. If ax + by is an eigenvector of 7, prove thata = 0 orb = 0. 


. Let T: S > V bea linear transformation such that every nonzero element of S is an eigenvector. 


Prove that there exists a scalar c such that T(x) = cx. In other words, the only transformation 
with this property is a scalar times the identity. [Hint: Use Exercise 11.] 
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4.5 The finite-dimensional case. Characteristic polynomials 


If dim V = n, the problem of finding the eigenvalues of a linear transformation T: V > V 
can be solved with the help of determinants. We wish to find those scalars A such that the 
equation 7(x) = Ax has a solution x with x # O. The equation T(x) = Ax can be written 
in the form 


(AI — T)(x) oy. O, 


where / is the identity transformation. If we let T, = AJ — T, then Ais an eigenvalue if and 
only if the equation 


(4.6) T,(x) = O 


has a nonzero solution x, in which case 7, is not invertible (because of Theorem 2.10). 
Therefore, by Theorem 2.20, a nonzero solution of (4.6) exists if and only if the matrix of 
T, is singular. If A is a matrix representation for 7, then AJ — A is a matrix representation 
for T,. By Theorem 3.13, the matrix AJ — A is singular if and only if det (AJ — A) = 0. 
Thus, if A is an eigenvalue for 7 it satisfies the equation 


(4.7) det (AI — A) =0. 


Conversely, any A in the underlying field of scalars which satisfies (4.7) is an eigenvalue. 
This suggests that we should study the determinant det (AJ — A) as a function of A. 


THEOREM 4.4. If A isanyn X n matrix and if lis then X n identity matrix, the function f 
defined by the equation 
(A) = det (AJ — A) 


is a polynomial in A of degree n. Moreover, the term of highest degree is 4", and the constant 
term is f(0) = det (—A) = (—1)” det A. 


Proof. The statement f(0) = det (—A) follows at once from the definition of f/ We 
prove that fis a polynomial of degree n only for the case n ¢ 3. The proof in the general 
case can be given by induction and is left as an exercise. (See Exercise 9 in Section 4.8.) 

For n = 1 the determinant is the linear polynomial f(A) = 4 — a,,;. For n = 2 we have 


A— ay — Ayo 


det (AJ — A) = = (A — ay,)(A — Qg2) — 4221 


— Qo A— Qe0 


= 2® — (Ay, + Aoa)A + (Gy1Q02 — 12421). 
a quadratic polynomial in 4. For n = 3 we have 
A— ay —Ay2 —aj3 


det (AI — A) = — Ag, A — Ago — Ao 


— a3 — Azo A — agg 
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A — Aog — Ao 


= (A — a) 


— Age A— 433 


The last two terms are linear polynomials in 4. The first term is a cubic polynomial, the 
term of highest degree being A?. 


DEFINITION. Jf A is ann X n matrix the determinant 
f(A) = det (AI — A) 


is called the characteristic polynomial of A. 


The roots of the characteristic polynomial of A are complex numbers, some of which 
may be real. If we let F denote either the real field R or the complex field C, we have the 
following theorem. 


THEOREM 4.5. Let T: V— V be a linear transformation, where V has scalars in F, and 
dim V =n. Let A be a matrix representation of T. Then the set of eigenvalues of T consists 
of those roots of the characteristic polynomial of A which lie in F. 


Proof. The discussion preceding Theorem 4.4 shows that every eigenvalue of T satisfies 
the equation det (AJ — A) = 0 and that any root of the characteristic polynomial of 4 
which lies in F is an eigenvalue of T. 


The matrix A depends on the choice of basis for V, but the eigenvalues of T were defined 
without reference to a basis. Therefore, the set of roots of the characteristic polynomial of 
A must be independent of the choice of basis. More than this is true. Ina later section we 
shall prove that the characteristic polynomial itself is independent of the choice of basis. 
We turn now to the problem of actually calculating the eigenvalues and eigenvectors in the 
finite-dimensional case. 


4.6 Calculation of eigenvalues and eigenvectors in the finite-dimensional case 


In the finite-dimensional case the eigenvalues and eigenvectors of a linear transformation 
T are also called eigenvalues and eigenvectors of each matrix representation of T. Thus, the 
eigenvalues of a square matrix A are the roots of the characteristic polynomial f(A) = 
det (AJ — A). The eigenvectors corresponding to an eigenvalue A are those nonzero 
vectors X = (x,,...,X,) regarded asm xX 1 column matrices satisfying the matrix equation 


AX=AX, or (AI-A)X=O. 


This is a system of n linear equations for the components x,,...,%X,. Once we know A 
we can obtain the eigenvectors by solving this system. We illustrate with three examples 
that exhibit different features. 
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EXAMPLE 1. A matrix with all its eigenvalues distinct. The matrix 


2 1 1 
A=! 2 3 4 
—1 -—1 -2 
has the characteristic polynomial 
A—-2 -! —1 
det(AT— A) =det] —2 A-—3 —-4 | =(A—1)44+ 10 — 3), 
1 1 A+2 


so there are three distinct eigenvalues: 1, —1 ,and3. To find the eigenvectors corresponding 
to A = 1 we solve the system AX = X, or 


This gives us 
2x, + Xo + X3 = X1 
2X, + 3x, + 4x5 = Xe 


—X,~— X_y — 2X3 = Xs, 
which can be rewritten as 
Xi, + Xo + x, =0 


2x, + 2x. + 4x, = 0 


—xX; — X_, — 3x, = 0. 


Adding the first and third equations we find x, = 0, and all three equations then reduce to 
X; + X, =0. Thus the eigenvectors corresponding to A = 1 are X = t(1, —1, 0), where 
tis any nonzero scalar. 

By similar calculations we find the eigenvectors X = ¢(0, 1 , —1) corresponding to A = 
—1, and X = ¢(2, 3, —1) corresponding to 4 = 3, with ¢ any nonzero scalar. Since the 
eigenvalues are distinct the corresponding eigenvectors (1, —1 , 0), (0,1, —1), and (2, 3, —1) 
are independent. The results can be summarized in tabular form as follows. In the third 
column we have listed the dimension of the eigenspace E(A). 


Eigenvalue A Eigenvectors dim E(A) 
1 71, —1,0), +40 1 
—1 0,1, -—-1), +#0 1 


3 (2,3, —1), t #0 1 


n 
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EXAMPLE 2. A matrix with repeated eigenvalues. The matrix 


2 —1 l 
A=|0 3 -1 
2 1 3 
has the characteristic polynomial 
A—-2 1 —1 
det(AT— A)=det} 0 A-3 1 = (A — 2)(A — 2)(A — 4). 
—2 —l1 A—3 


The eigenvalues are 2, 2, and 4. (We list the eigenvalue 2 twice to emphasize that it is a 
double root of the characteristic polynomial.) To find the eigenvectors corresponding to 
A = 2 we solve the system AX = 2X, which reduces to 


—X,+ x, =0 
X, — X, = 0 
2X, +X, +x, = 
This has the solution x, = x3 = —x, so the eigenvectors corresponding to A = 2 are 


t(—1,1,1), where t+ #0. "Similarly we find the eigenvectors ¢t(1, —1, 1) corresponding 
to the eigenvalue A = 4. The results can be summarized as follows: 


Eigenvalue Figenvectors dim E(A) 
Zid t(—1,1,1), ¢+#090 ] 
4 t1,—1,1), t#0 ] 


EXAMPLE 3. Another matrix with repeated eigenvalues. The matrix 


21 1 
A=|2 3 2 
3 3 4 


has the characteristic polynomial (A — 1)(A — 1)(A — 7). When A = 7 the system AX = 
7X becomes 


3X, =r. Xo aa X3 — 0 
—2x, + 4x, = 2X3 — 0 
—3x, a 3Xo + 3X5 = 0 ° 
This has the solution x, = 2x,, x3 = 3x,, so the eigenvectors corresponding to A = 7 are 


t(1, 2,3), where ¢ #0. For the eigenvalue 2 = 1, the system AX = X consists of the 
equation 
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X, +X, + x, = 0 


repeated three times. To solve this equation we may take x, = a, x, = b, where a and b 
are arbitrary, and then take x, = —a — b. Thus every eigenvector corresponding to A = 1 
has the form 

(a,b, —a — b) = a(1, 0, —1) + 5(0, 1, —1), 


where a ~ 0 orb #0. This means that the vectors (1, 0, —1) and (0, 1, —1) form a basis 
for E(1). Hence dim E(A) = 2 when A = 1. The results can be summarized as follows: 


Eigenvalue Eigenvectors dim E(A) 
7 t(1,2,3), +40 | 
1, 1 a(1,0, —1) + 6(0, 1, —1), a, b not both 0. 2 


Note that in this example there are three independent eigenvectors but only two distinct 
eigenvalues. 


4.7 Trace of a matrix 


Let f(A) be the characteristic polynomial of ann X n matrix A. We denote the x roots of 
f(A) by 4,,...,4,, with each root written as often as its multiplicity indicates. Then we 
have the factorization 


f(A) = A — a) A Ay). 
We can also write f(A) in decreasing powers of 4 as follows, 
SA) =A +e, dr tte + aat oy. 


Comparing this with the factored form we find that the constant term cy and thecoefficient of 
A" are given by the formulas 


Co = (—1)"A, °° +A, and Cay = (A, He + A,). 
Since we also have cy = (—1)” det A, we see that 
Ay: +A, = det A. 
That is, the product of the roots of the characteristic polynomial of A is equal to the determinant 


of A. 
The sum of the roots of f(A) is called the trace of A, denoted by tr A. Thus, by definition, 


n 
i=1 


The coefficient of A"! is given by c,_, = —tr A. We can also compute this coefficient 
from the determinant form for f(A) and we find that 


Ss aed — (ay oe? op Ann) « 
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(A proof of this formula is requested in Exercise 12 of Section 4.8.) The two formulas for 
C,-1 show that 


n 
tr A = > a;;- 
i=1 


That is, the trace of A is also equal to the sum of the diagonal elements of A. 

Since the sum of the diagonal elements is easy to compute it can be used as a numerical 
check in calculations of eigenvalues. Further properties of the trace are described in the 
next set of exercises. 


4.8 Exercises 


Determine the eigenvalues and eigenvectors of each of the matrices in Exercises 1 through 3. 
Also, for each eigenvalue 4 compute the dimension of the eigenspace E(A). 


1 0 1 1 1 0 1 1 
1. (a) | (b) |: (c) | (d) | 
0 1 0 1 1 1 1 1 


1 a cos 9 -—sin 0 
ae ,a>o0,b>0. 3. : 
b 1 sin@  cos@ 
0 1 0 -i 1 0 
4. The matrices P, = , P= , P= occur in the quantum 
1 0 i 0 0 —-1 


mechanical theory of electron spin and are called Pauli spin matrices, in honor of the physicist 
Wolfgang Pauli (1900-1958). Verify that they all have eigenvalues 1 and —1. Then determine 
all 2 x 2 matrices with complex entries having the two eigenvalues 1 and —1. 

5. Determine all 2 x 2 matrices with real entries whose eigenvalues are (a) real and distinct, (b) 
real and equal, (c) complex conjugates. 

6. Determine a, b, c, d, e, f, given that the vectors (1, 1, 1), (1, 0, —1), and (1, —1,, 0) are eigen- 
vectors of the matrix 11 


abc}. 
def 


7. Calculate the eigenvalues and eigenvectors of each of the following matrices. Also, compute 
the dimension of the eigenspace E(A) for each eigenvalue 4. 


1 0 0 2 1 3 5 —6 —6 
(2)|/-3 1 #O|, @l1 2 3], @l—-1 4.— 2). 
4-7 1 3 3 20 3: 26: 44 


8. Calculate the eigenvalues of each of the five matrices 


00 1 0 1 0 0 0 0 1 0 0 

0 0 0 1 0 1 0 0 100 0 
(a) ’ (b) ’ (c) ’ 

100 0 0 0 -1l 0 0 0 0 1 

0 1 0 0 0 0 0 —-l 0 0 1 0 
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0 -i 0 0 i 0 oO 0 
i 0 0 0 0 - 0 0 

(d) » ©) 
0 0 0 -i 0 1 0 
0 i 0 0 0 O -1 


These are called Dirac matrices in honor of Paul A. M. Dirac (1902-  ), the English 
physicist. They occur in the solution of the relativistic wave equation in quantum mechanics. 
9. If Aand Baren x n matrices, with B a diagonal matrix, prove (by induction) that the deter- 
minant f(A) = det (AB — A) is a polynomial in 4 with f(0) = (—1)" det A, and with the 
coefficient of 4” equal to the product of the diagonal entries of B. 
10. Prove that a square matrix A and its transpose A’ have the same characteristic polynomial. 
11. If A and Bare n X n matrices, with A nonsingular, prove that 4B and BA have the same set 
of eigenvalues. Note: It can be shown that AB and BA have the same characteristic poly- 
nomial, even if A is singular, but you are not required to prove this. 
12. Let A be ann X n matrix with characteristic polynomial f(A). Prove (by induction) that the 
coefficient of 4"~1 in f(A) is —tr A. 
13. Let A and B ben x n matrices with det A = det Band tr A =tr B. Prove that A and B have 
the same characteristic polynomial if n = 2 but that this need not be the case if n > 2. 
14. Prove each of the following statements about the trace. 


(a) tr(A + B) =trA +trB. 
(b) tr (cA) =ctrA. 

(c) tr (AB) = tr (BA). 

(d) trAé =trA. 


4.9 Matrices representing the same linear transformation. Similar matrices 


In this section we prove that two different matrix representations of a linear trans- 
formation have the same characteristic polynomial. To do this we investigate more closely 
the relation between matrices which represent the same transformation. 

Let us recall how matrix representations are defined. Suppose 7: V > W is a linear 
mapping of an n-dimensional space V into an m-dimensional space W. Let (e,,..., e,) 
and (w,,...,W,) be ordered bases for V and W respectively. The matrix representation 
of T relative to this choice of bases is the m xX n matrix whose columns consist of the com- 
ponents of T(e,),... ,7(e,) relative to the basis (w,,...,W,,). Different matrix represen- 
tations arise from different choices of the bases. 

We consider now the case in which V = W, and we assume that the same ordered basis 
(e€,,...,@,) 18 used for both V and W. Let A = (a,,) be the matrix of T relative to this 
basis. This means that we have 


(4.8) T(e,) = Da,e; for k=1,2,...,n. 
i=l 


Now choose another ordered basis (u,,..., u,) for both Vand W and let B = (5,,) be the 
matrix of T relative to this new basis. Then we have 
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(4.9) T(u;)=>b,ju, for j=1,2,...,n. 
k=1 
Since each u; is in the space spanned by e,,..., e, we can write 
(4.10) t= eee, for pS isla, 
k=1 


for some set of scalars c,;. Then X n matrix C = (c,;) determined by these scalars is non- 
singular because it represents a linear transformation which maps a basis of V onto another 
basis of V. Applying T to both members of (4.10) we also have the equations 


(4.11) Ttu;)=>c,;T(e,) for j=1,2,...,n. 
k=1 


The systems of equations in (4.8) through (4.11) can be written more simply in matrix 
form by introducing matrices with vector entries. Let 


| aa (are and U = [u,,...,u,] 


be 1 x nm row matrices whose entries are the basis elements in question. Then the set of 
equations in (4.10) can be written as a single matrix equation, 


(4.12) USEC. 
Similarly, if we introduce 
E’ = [T(e,),..., T(e,)] and U’ = [T(u,),..., T(u,)], 
Equations (4.8), (4.9), and (4.11) become, respectively, 
(4.13) BE =A; U’ = UB, Ul = EAC. 
From (4.12) we also have 
b= CC, 


To find the relation between A and B we express U’ in two ways in terms of U. From 
(4.13) we have 


and 
U’ = E’C = EAC = UC"1AC. 


Therefore UB = UC“1AC. But each entry in this matrix equation is a linear combination 
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of the basis vectors u,,...,u,. Since the u; are independent we must have 
B=C AC. 
Thus, we have proved the following theorem. 


THEOREM 4.6. If twon x n matrices A and B represent the same linear transformation T, 
then there is a nonsingular matrix C such that 


B=C AC. 
Moreover, if A is the matrix of T relative to a basis E = [e,,..., @,] and if B is the matrix 
of T relative to a basis U = [u,,...,u,], then for C we can take the nonsingular matrix 


relating the two bases according to the matrix equation U = EC. 
The converse of Theorem 4.6 is also true. 


THEOREM 4.7. Let A and B be two n X n matrices related by an equation of the form 
B= CAC, where C is a nonsingular n x n matrix. Then A and B represent the same 
linear transformation. 


Proof. Choose a basis E = [e,,..., e,] for an n-dimensional space V. Let u,,..., Uy 
be the vectors determined by the equations 


(4.14) u;=D>c je for j=1,2,...,n, 
1 


where the scalars c,, are the entries of C. Since C is nonsingular it represents an invertible 
linear transformation, so U = [u,,..., u,] is also a basis for V, and we have U = EC. 

Let T be the linear transformation having the matrix representation A relative to the basis 
E, and let S be the transformation having the matrix representation B relative to the basis U. 
Then we have 


(4.15) T(e,) a > Qi; for k a 1, Zs coe g ll 
t=1 

and 

(4.16) S(uj)=> b,j, for j=1,2,...,n. 
k=1 


We shall prove that S = T by showing that T(u,) = S(u,) for each /. 
Equations (4.15) and (4.16) can be written more simply in matrix form as follows, 
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[T(e,),..., T(e,)] = EA,  [S(u,),..., S(u,)] = UB. 
Applying T to (4.14) we also obtain the relation T(u;) = > c,;T(e,), or 
[T(u,),..., T(u,)] = EAC. 
But we have 
UB = ECB = EC(C1AC) = EAC, 


which shows that 7(u;) = S(u;) for each j. Therefore T(x) = S(x) for each x in V, so T = 
S. In other words, the matrices A and B represent the same linear transformation. 


DEFINITION. Zwo n Xn matrices A and B are called similar if there is a nonsingular 
matrix C such that B = CAC. 


Theorems 4.6 and 4.7 can be combined to give us 


THEOREM 4.8. Twon X n matrices are similar if and only if they represent the same linear 
transformation. 


Similar matrices share many properties. For example, they have the same determinant 
since 


det (C-1AC) = det (C~1)(det A)(det C) = det A. 


This property gives us the following theorem. 


THEOREM 4.9. Similar matrices have the same characteristic polynomial and therefore the 
same eigenvalues. 


Proof. \f A and B are similar there is a nonsingular matrix C such that B= C-lAC. 
Therefore we have 


AI — B= AI — C1AC = ACHUC — C14AC = CAI — ANC. 
This shows that AJ — B and AJ — A are similar, so det (AJ — B) = det (AJ — A). 


Theorems 4.8 and 4.9 together show that all matrix representations of a given linear 
transformation T have the same characteristic polynomial. This polynomial is also called 
the characteristic polynomial of T. 

The next theorem is a combination of Theorems 4.5, 4.2, and 4.6. In Theorem 4.10, F 
denotes either the real field R or the complex field C. 
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THEOREM 4.10. Let T: V— V be a linear transformation, where V has scalars in F, and 
dim V = n. Assume that the characteristic polynomial of T has n distinct roots A,,..., 4, in 
F. Then we have: 

(a) The corresponding eigenvectors u,,...,u, form a basis for V. 

(b) The matrix of T relative to the ordered basis U = [u,,..., u,] is the diagonal matrix 

A having the eigenvalues as diagonal entries: 


n 


A = diag (A,,...,4,). 
(c) If A is the matrix of T relative to another basis E = [e,,...,¢,], then 
A=CTAC, 
where C is the nonsingular matrix relating the two bases by the equation 
U= EC. 


Proof. By Theorem 4.5 each root A; is an eigenvalue. Since there are n distinct roots, 
Theorem 4.2 tells us that the corresponding eigenvectors u,,...,u, are independent. 
Hence they form a basis for V. This proves (a). Since T(u;) = A,u; the matrix of T relative 
to U is the diagonal matrix A, which proves (b). To prove (c) we use Theorem 4.6. 


Note: The nonsingular matrix C in Theorem 4.10 is called a diagonalizing matrix. If 
(€,,..., &,) is the basis of unit coordinate vectors (J,,...,/,), then the equation U = 
EC in Theorem 4.10 shows that the kth column of C consists of the components of the 
eigenvector u, relative to (),..., In). 


If the eigenvalues of A are distinct then A is similar to a diagonal matrix. If the eigen- 
values are not distinct then A still might be similar to a diagonal matrix. This will happen 
if and only if there are k independent eigenvectors corresponding to each eigenvalue of 
multiplicity k. Examples occur in the next set of exercises. 


4.10 Exercises 


1 1 1 0 
1. Prove that the matrices and | have the same eigenvalues but are not similar. 
0 1 0 1 


2. In each case find a nonsingular matrix C such that C~*AC is a diagonal matrix or explain why 
no such C exists. 


1 0 1 2 21 21 
@ =| | a=] | oa=| | @4=| | 
1 3 5 4 -1 4 -~1 0 


3. Three bases in the plane are given. With respect to these bases a point has components (xj, X2)> 
(y1, Y2), and (z,, Z,), respectively. Suppose that [y1, yo] = [x,, xe]A, [Z,, Z.] = [%1, x,]B, and 
[Z1, Zo] = [y1, yelIC, where A, B, Care 2 x 2 matrices. Express C in terms of A and B. 
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4. In each case, show that the eigenvalues of A are not distinct but that A has three independent 
eigenvectors. Find a nonsingular matrix C such that C~'AC is a diagonal matrix. 


00 1 i -1 -! 
(a) A=|]0 1 Of, (b) A = 1 3 1 
100 -i -!l | 


5. Show that none of the following matrices is similar to a diagonal matrix, but that each is similar 


A 0 
to a triangular matrix of the form , where 4 is an eigenvalue. 
1 A 


2 -1 2 1 
| | | | 
0 2 —1 4 


6. Determine the eigenvalues and eigenvectors of the matrix | 0O 0 1 | and thereby show 
that it is not similar to a diagonal matrix. 


0 -1 0 


7. (a) Prove that a square matrix A is nonsingular if and only if 0 is not an eigenvalue of A. 
(b) If A is nonsingular, prove that the eigenvalues of A~ are the reciprocals of the eigenvalues 
of A. 


8. Given ann x n matrix A with real entries such that A? = —J. Prove the following statements 
about A. 
(a) A is nonsingular. 
(b) 1 is even. 
(c) A has no real eigenvalues. 
(d) det A = 1. 


5) 


EIGENVALUES OF OPERATORS ACTING ON 
EUCLIDEAN SPACES 


5.1 Eigenvalues and inner products 


This chapter describes some properties of eigenvalues and eigenvectors of linear trans- 
formations that operate on Euclidean spaces, that is, on linear spaces having an inner 
product. We recall the fundamental properties of inner products. 

In a real Euclidean space an inner product (x, y) of two elements x and y is a real number 
satisfying the following properties: 


(1) (xy) =.) (symmetry) 
(2) (x + z, y) = (x, y) + (2, y) (linearity) 

(3) (cx, y) = c(x, y) (homogeneity) 
(4) (x%,x)>0 if x#O (positivity). 


In a complex Euclidean space the inner product is a complex number satisfying the same 
properties, with the exception that symmetry is replaced by Hermitian symmetry, 


(1’) (x,y) = (,%), 


where the bar denotes the complex conjugate. In (3) the scalar c is complex. From (1’) 
and (3) we obtain 


(3’) (x, cy) = e(x, y), 


which tells us that scalars are conjugated when taken out of the second factor. Taking 
x = y in (1’) we see that (x, x) is real so property (4) is meaningful if the space is complex. 
When we use the term Euclidean space without further designation it is to be understood 
that the space can be real or complex. Although most of our applications will be to finite- 
dimensional spaces, we do not require this restriction at the outset. 
The first theorem shows that eigenvalues (if they exist) can be expressed in terms of the 
inner product. 


THEOREM 5.1. Let E be a Euclidean space, let V be a subspace of E, and let T: V—> E bea 
linear transformation having an eigenvalue 4 with a corresponding eigenvector x. Then we have 


(5.1) , — (TO), ») © 
) (x, x) 
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Proof. Since T(x) = Ax we have 
(T(x), x) = (Ax, x) = A(x, x). 
Since x # O we can divide by (x, x) to get (5.1). 


Several properties of eigenvalues are easily deduced from Equation (5.1). For example, 
from the Hermitian symmetry of the inner product we have the companion formula 


—— (x, T(x)) 
(5.2) = —- 


for the complex conjugate of A. From (5.1) and (5.2) we see that A is real (A = A) if and 
only if (7(x), x) is real, that is, if and only if 


(T(x), x) = (x, T(x)) for the eigenvector x. 


(This condition is trivially satisfied in a real Euclidean space.) Also, A is pure imaginary 
(A = —A) if and only if (T(x), x) is pure imaginary, that is, if and only if 


(T(x), x) = —(x, T(x)) for the eigenvector x. 


5.2 Hermitian and skew-Hermitian transformations 


In this section we introduce two important types of linear operators which act on Euclid- 
ean spaces. These operators have two categories of names, depending on whether the 
underlying Euclidean space has a real or complex inner product. In the real case the trans- 
formations are called symmetric and skew-symmetric. In the complex case they are called 
Hermitian and skew-Hermitian. These transformations occur in many different applications. 
For example, Hermitian operators on infinite-dimensional spaces play an important role in 
quantum mechanics. We shall discuss primarily the complex case since it presents no added 
difficulties. 


DEFINITION. Let E be a Euclidean space and let V be a subspace of E. A linear trans- 
formation T: V — E is called Hermitian on V if 


(T(x), y)=(x%,T(Q))) forall x andyinV. 


Operator T is called skew-Hermitian on V if 


(T(x), y) = —(x,T(y)) ~—s for all x andy in V. 


In other words, a Hermitian operator T can be shifted from one factor of an inner product 
to the other without changing the value of the product. Shifting a skew-Hermitian operator 
changes the sign of the product. 


Note: As already mentioned, if E is a real Euclidean space, Hermitian transformations 
are also called symmetric; skew-Hermitian transformations are called skew-symmetric. 
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EXAMPLE 1. Symmetry and skew-symmetry in the space C(a, b). Let C(a, b) denote the 
space of all real functions continuous on a closed interval [a, b], with the real inner product 


(f, 8) = |’ Fat) at. 


Let V be a subspace of C(a, b). If T: V — C(a, b)isa linear transformation then ( f, T(g)) = 
f° f(t)Tg(t) dt, where we have written Tg(t) for T(g)(t). Therefore the conditions for sym- 
metry and skew-symmetry become 


(5.3) [’ F@Te@) — e(OT/(D} dt =0 if Tis symmetric, 
and 
(5.4) (n {f(t)T g(t) + g(Tf()} dt =0 if T is skew-symmetric. 


EXAMPLE 2. Multiplication by a fixed function. In the space C(a, b) of Example 1, choose 
a fixed function p and define T(f) = pf, the product of pand/f. For this 7, Equation (5.3) 
is satisfied for all f and g in C(a, b) since the integrand is zero. Therefore, multiplication 
by a fixed function is a symmetric operator. 


EXAMPLE 3. The differentiation operator. In the space C(a, b) of Example 1, let V be the 
subspace consisting of all functions f which have a continuous derivative in the open interval 
(a, b) and which also satisfy the boundary condition f(a) = f(b). Let D: V > C(a, b) be the 
differentiation operator given by D(f) =f’. It is easy to prove that D is skew-symmetric. 
In this case the integrand in (5.4) is the derivative of the product fg, so the integral is equal 
to 


ie (fg)'(t) dt = f(b)g(b) — f(a)g(a). 


Since both fand g satisfy the boundary condition, we have f(b)g(b) — f(a)g(a) = 0. Thus, 
the boundary condition implies skew-symmetry of D. The only eigenfunctions in the sub- 
space V are the constant functions. They belong to the eigenvalue 0. 


EXAMPLE 4. Sturm-Liouville operators. This example is important in the theory of linear 
second-order differential equations. We use the space C(a, 6) of Example | once more and 
let V be the subspace consisting of all f which have a continuous second derivative in [a, b] 
and which also satisfy the two boundary conditions 


(5.5) Pla)f(a)=0, pd) f(b) =0, 


where p is a fixed function in C(a, b) with a continuous derivative on [a, b]. Let g be another 
fixed function in C(a, b) and let T: V> C(a, b) be the operator defined by the equation 


T(f) = (f') + 74. 
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This is called a Sturm-Liouville operator. To test for symmetry we note that fT(g) — 
eT(f) = f(g’) — g(pf')’. Using this in (5.3) and integrating both f° f- (pg’)’ dt and 
J? g- (pf) dt by parts, we find 


i {fT(g) — gT(f)} dt = fps’ — .? pgf’ dt — gpf' P+ ie pf'g’ dt = 0, 


since both f and g satisfy the boundary conditions (5.5). Hence T is symmetric on V. The 
eigenfunctions of T are those nonzero f which satisfy, for some real A, a differential equation 
of the form 


(pf') + af = +f 


on [a, b], and also satisfy the boundary conditions (5.5). 


5.3 Eigenvalues and eigenvectors of Hermitian and skew-Hermitian operators 


Regarding eigenvalues we have the following theorem. 


THEOREM 5.2. Assume T has an eigenvalue 4. Then we have: 
(a) If T is Hermitian, A is real? A =i. ; 
(b) If T is skew-Hermitian, 4 is pure imaginary: 4 = —/. 


Proof. Let x be an eigenvector corresponding to A. Then we have 


(T(x), x) and A a> (x, T(x)) 
(x, x) (x, x) 


If Tis Hermitian we have (T(x), x) = (x, T(x))soA = A. If Tis skew-Hermitian we have 
(T(x), x) = —(x, T(x)) sodA = —A. 


Note: If Tis symmetric, Theorem 5.2 tells us nothing new about the eigenvalues of T 
since all the eigenvalues must be real if the inner product is real. If Tis skew-symmetric, 
the eigenvalues of T must be both real and pure imaginary. Hence all the eigenvalues 
of a skew-symmetric operator must be zero (if any exist). 


A= 


5.4 Orthogonality of eigenvectors corresponding to distinct eigenvalues 


Distinct eigenvalues of any linear transformation correspond to independent eigen- 
vectors (by Theorem 4.2). For Hermitian and skew-Hermitian transformations more is 
true. 


THEOREM 5.3. Let T be a Hermitian or skew-Hermitian transformation, and let hand u 
be distinct eigenvalues of T with corresponding eigenvectors x and y. Then x and y are 
orthogonal; that is, (x, y) =0. 


Proof. We write T(x) = Ax, T(y) = py and compare the two inner products (T(x), y) 
and (x, T(y)). We have 


(T(x), y) = x,y) =A, y) and (x, TY) = (x, wy) = AG, J). 
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If 7 is Hermitian this gives us A(x, y) = A(x, y) = w(x, y) since uw = f@. Therefore 
(x, y) = 0 since A ¥ w. If T is skew-Hermitian we obtain A(x, y) = —fi(x, y) = u(x, y) 
which again implies (x, y) = 0. 


EXAMPLE. We apply Theorem 5.3 to those nonzero functions which satisfy a differential 
equation of the form 


(5.6) (pf) + qf = af 


on an interval [a, b], and which also satisfy the boundary conditions p(a) f(a) = p(b) f(b) = 
0. The conclusion is that any two solutions f and g corresponding to two distinct values of 
A are orthogonal. For example, consider the differential equation of simple harmonic 
motion, 

f" + kf =0 


on the interval [0,7], where kK #0. This has the form (5.6) with p=1, q=0, and 
A = —k?*. All solutions are given by f(t) = c, cos kt + c, sin kt. The boundary condition 
(0) = Oimplies c, = 0. The second boundary condition, f(z) = 0, implies c, sinka = 0. 
Since c. ¥ 0 for a nonzero solution, we must have sin kw = 0, which means that k is an 
integer. In other words, nonzero solutions which satisfy the boundary conditions exist if 
and only if k is an integer. These solutions are f(t) = sinnt, n= +1,+2,.... The 
orthogonality condition implied by Theorem 5.3 now becomes the familiar relation 


Tv 
(i sin nt sin mt dt = 0 


if m? and n? are distinct integers. 


5.5 Exercises 


1. Let E be a Euclidean space, let V be a subspace, and let T: V > E be a given linear trans- 
formation. Let 4 be a scalar and x a nonzero element of V. Prove that 4 is an eigenvalue of T 
with x as an eigenvector if and only if 


(T(x), y) = A(x, y) ‘for every yin E. 


2. Let T(x) = cx for every x in a linear space V, where c is a fixed scalar. Prove that T is sym- 
metric if V is a real Euclidean space. 

3. Assume T: V > V is a Hermitian transformation. 
(a) Prove that T” is Hermitian for every positive integer n, and that T~* is Hermitian if T is 
invertible. 
(b) What can you conclude about JT” and T~ if T is skew-Hermitian ? 

4. Let T,: V > Eand T,: V > E be two Hermitian transformations. 
(a) Prove that aT, + bT, is Hermitian for all real scalars a and b. 
(b) Prove that the product (composition) 7,7, is Hermitian if 7, and 7, commute, that is, if 
T,T, = T,T,. 

5. Let V = V,(R) with the usual dot product as inner product. Let T be a reflection in the xy- 
plane; that is, let Tj) =i, Tj) =j, and T(k) = —k. Prove that T is symmetric. 


6. 


10. 
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Let C(O, 1) be the real linear space of all real functions continuous on [0, 1] with inner product 
(f,g) = fi fOg@ dt. Let V be the subspace of all f such that J} f(t)dt =0. Let T: V—> 
C(0, 1) be the integration operator defined by Tf(x) = Jé f(t) dt. Prove that T is skew-sym- 
metric. 


. Let V be the real Euclidean space of all real polynomials with the inner product (f/f, g) = 


ft, f@g() dt. Determine which of the following transformations T: V > V is symmetric or 
skew-symmetric: 

(a) Tf(x) = f(—). (c) Tf(x) = f(x) + f(—). 

(b) Tf(x) =f@)f(-»). (d) Tf) =f) —f(-%). 


. Refer to Example 4 of Section 5.2. Modify the inner product as follows: 


(f.3) = [> fOgnw(o at, 


where w is a fixed positive function in C(a, 6). Modify the Sturm-Liouville operator T by 
writing 


T(f) = (pf + af 


Prove that the modified operator is symmetric on the subspace V. 


. Let V be a subspace of a complex Euclidean space E. Let T: V > E bea linear transformation 


and define a scalar-valued function Q on V as follows: 


Q(x) = (T(x), x) for all x in V. 


(a) If T is Hermitian on V, prove that Q(x) is real for all x. 

(b) If T is skew-Hermitian, prove that Q(x) is pure imaginary for all x. 

(c) Prove that Q(tx) = ttQ(x) for every scalar ¢. 

(d) Prove that Q(x + y) = Q(x) + Q(y) + (T@), y) + (TV), x), and find a corresponding 
formula for Q(x + ty). 

(e) If Q(x) = 0 for all x prove that T(x) = O for all x. 

(f) If Q(x) is real for all x prove that T is Hermitian. [Hint: Use the fact that Q(x + ty) 
equals its conjugate for every scalar ¢.] 

This exercise shows that the Legendre polynomials (introduced in Section 1.14) are eigen- 
functions of a Sturm-Liouville operator. The Legendre polynomials are defined by the equation 


1 
2”"n! 


P(t) =—— f™@), where f,() = (2 — 1)”. 


(a) Verify that ( — 1) f),(0) = 2ntf,(). 
(b) Differentiate the equation in (a) + 1 times, using Leibniz’s formula (see p. 222 of Volume 
I) to obtain 


(22 — If 2(r) + ttn + YK HYD) + nn + DMO = 2nef VEO + nn + D/MC. 
(c) Show that the equation in (b) can be rewritten in the form 


(a — DP OY =n + IPC). 


This shows that P,(¢) is an eigenfunction of the Sturm-Liouville operator T given on the 
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interval [—1 , 1] by T(f) = (pf ’)’, where p(t) = —1. The eigenfunction P,,(s) belongs to 
the eigenvalue 4 = n(n + 1). In this example the boundary conditions for symmetry are 
automatically satisfied since p(1) = p(—1) = 0. 


5.6 Existence of an orthonormal set of eigenvectors for Hermitian and skew-Hermitian 
operators acting on finite-dimensional spaces 


Both Theorems 5.2 and 5.3 are based on the assumption that T has an eigenvalue. As 
we know, eigenvalues need not exist. However, if 7 acts on a finite-dimensional complex 
space, then eigenvalues always exist since they are the roots of the characteristic polynomial. 
If T is Hermitian, all the eigenvalues are real. If T is skew-Hermitian, all the eigenvalues 
are pure imaginary. 

We also know that two distinct eigenvalues belong to orthogonal eigenvectors if T is 
Hermitian or skew-Hermitian. Using this property we can prove that T has an orthonormal 
set of eigenvectors which spans the whole space. (We recall that an orthogonal set is called 
orthonormal if each of its elements has norm 1.) 


THEOREM 5.4. Assume dim V =n and let T: V—>V be Hermitian or skew-Hermitian. 
Then there exist n eigenvectors u,,...,U, of T which form an orthonormal basis for V. 
Hence the matrix of T relative to this basis is the diagonal matrix A = diag (A,,...,4,), 
where A, is the eigenvalue belonging to u, . 


Proof. We use induction on the dimension n. If n = 1, then T has exactly one eigen- 
value. Any eigenvector u, of norm | is an orthonormal basis for V. 

Now assume the theorem is true for every Euclidean space of dimension nm — 1. To 
prove it is also true for V we choose an eigenvalue A, for T and a corresponding eigenvector 
u, of norm 1. Then 7(u,) = A,u, and ||u,|| = 1. Let S be the subspace spanned by u,. 
We shall apply the induction hypothesis to the subspace S+ consisting of all elements in V 
which are orthogonal to u,, 


St = {x|x eV, (x, u,) = 0}. 


To do this we need to know that dim St = n — | and that T maps S+ into itself. 

From Theorem 1.7(a) we know that u, is part of a basis for V, say the basis 
(u,,U,,...,0U,). We can assume, without loss in generality, that this is an orthonormal 
basis. (If not, we apply the Gram-Schmidt process to convert it into an orthonormal 
basis, keeping uw, as the first basis element.) Now take any x in S+ and write 


X == XyUy + Xe +°°*° + X,Vy.- 
Then x, = (x, u,) = 0 since the basis is orthonormal, so x is in the space spanned by 
Vo,...,U,. Hencedim St =n—1. 


Next we show that T maps S“ into itself. Assume T is Hermitian. If x € S+ we have 


(T(x), U;) = (x, T(u,)) ae (x, Ayu) = A(x; uy) = (0, 
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so T(x)€S+. Since T is Hermitian on St we can apply the induction hypothesis to find 
that T has nm — 1 eigenvectors u,,...,u, Which form an orthonormal basis for S-. 
Therefore the orthogonal set u,,..., u, 1s an orthonormal basis for V. This proves the 
theorem if Tis Hermitian. A similar argument works if T is skew-Hermitian. 


5.7 Matrix representations for Hermitian and skew-Hermitian operators 


In this section we assume that V is a finite-dimensional Euclidean space. A Hermitian 
or skew-Hermitian transformation can be characterized in terms of its action on the 
elements of any basis. 


THEOREM 5.5. Let (€,,...,@,) be a basis for V and let T: V— V be a linear transfor- 
mation. Then we have: 

(a) T is Hermitian if and only if (T(e;), €:) = (e;, T(e;)) for all i and j. 

(b) T is skew-Hermitian if and only if (T(e;), e:) = —(e;, T(e:)) for all i and j. 


Proof. Take any two elements x and y in V and express each in terms of the basis 
elements, say x = > x,e; and y = > y,e;. Then we have 


XV(T(e;), &;)- 


n 
=1 


(TO), = (ExT), y) =Dx,(Te). ver) =D 


j=1i 


Similarly we find 
(x, T(y)) = x 2 *iFl€ss T(e;,)). 
jJ=17= 


Statements (a) and (b) following immediately from these equations. 


Now we characterize these concepts in terms of a matrix representation of T. 


THEOREM 5.6. Let (e,,...,€,) be an orthonormal basis for V, and let A = (a,;) be the 
matrix representation of a linear transformation T: V — V relative to this basis. Then we 
have: 

(a) T is Hermitian if and only if a;; = a,; for all i and j. 

(b) T is skew-Hermitian if and only if a;; = —4a;; for all i and j. 


Proof. Since A is the matrix of T we have T(e;) = >”_, 4,;e,. Taking the inner product 
of T(e;) with e; and using the linearity of the inner product we obtain 


(T(e;), e;) = (Save e =D aes , @;). 


But (€,, e;) = O unless k = i, so the last sum simplifies to a;,(e;, e;) = a,; since (e;, e;) = 1. 
Hence we have 


a;; = (T(e;), e:) for all i, 7. 
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Interchanging i and j, taking conjugates, and using the Hermitian symmetry of the inner 
product, we find 
Qj: = (e;, T(e,)) for all by. 


Now we apply Theorem 5.5 to complete the proof. 


5.8 Hermitian and skew-Hermitian matrices. The adjoint of a matrix 


The following definition is suggested by Theorem 5.6. 


DEFINITION. A square matrix A = (a;;) is called Hermitian if a,; = a;, for alli and j. 
Matrix A is called skew-Hermitian if a;; = —d,; for all i and j. 


Theorem 5.6 states that a transformation T on a finite-dimensional space V is Hermitian 
or skew-Hermitian according as its matrix relative to an orthonormal basis is Hermitian 
or skew-Hermitian. 

These matrices can be described in another way. Let A denote the matrix obtained by 
replacing each entry of A by its complex conjugate. Matrix A is called the conjugate of A. 
Matrix A is Hermitian if and only if it is equal to the transpose of its conjugate, A = A’. 
It is skew-Hermitian if A = — A’. 

The transpose of the conjugate is given a special name. 


DEFINITION OF THE ADJOINT OF A MATRIX. or any matrix A, the transpose of the conjugate, 
A*, is also called the adjoint of A and is denoted by A*. 


Thus, a square matrix A is Hermitian if A = A*, and skew-Hermitian if A = —A*. 
A Hermitian matrix is also called self-adjoint. 


Note: Much of the older matrix literature uses the term adjoint for the transpose of the 
cofactor matrix, an entirely different object. The definition given here conforms to the 
current nomenclature in the theory of linear operators. 


5.9 Diagonalization of a Hermitian or skew-Hermitian matrix 


THEOREM 5.7. Every n X n Hermitian or skew-Hermitian matrix A is similar to the 
diagonal matrix A = diag (A,,...,4,) of its eigenvalues. Moreover, we have 


A=C IMC, 
where C is a nonsingular matrix whose inverse is its adjoint, C~\ = C*. 


Proof. Let V be the space of n-tuples of complex numbers, and let (e,,..., @,) be the 
orthonormal basis of unit coordinate vectors. If x = > x,e; and y = > y,e, let the inner 
product be given by (x, y) = > x,j;. For the given matrix A, let T be the transformation 
represented by A relative to the chosen basis. Then Theorem 5.4 tells us that V has an 
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orthonormal basis of eigenvectors (u,,...,u,), relative to which T has the diagonal 
matrix representation A = diag (A,,...,4,), where A, is the eigenvalue belonging to u,. 
Since both A and A represent TJ they are similar, so we have A = C7!AC, where C = (c;;) 
is the nonsingular matrix relating the two bases: 


ls bee | [igs ss:3:€.163 


This equation shows that the jth column of C consists of the components of u, relative to 
(e,;,...,@,). Therefore c;; is the ith component of u;. The inner product of u; and uy; is 
given by 


n 
(u, 9 u;) — 2 CeiCei- 
k= 


Since {u,,...,u,,} 1S an orthonormal set, this shows that CC* = J, so C1 = C*. 


Note: The proof of Theorem 5.7 also tells us how to determine the diagonalizing 
matrix C. We find an orthonormal set of eigenvectors u,,...,u, and then use the 
components of u; (relative to the basis of unit coordinate vectors) as the entries of the jth 
column of C. 


2.2 
EXAMPLE |. The real Hermitian matrix A = ae has eigenvalues A, = 1 and A, = 6. 


The eigenvectors belonging to | are ¢(2, —1), t #0. Those belonging to 6 are ¢(1, 2), 
t 0. The two eigenvectors u, = ¢(2, —1) and u, = (1,2) with t= 1//5 form an 
orthonormal set. Therefore the matrix 


pe 
J5L-1 2 
is a diagonalizing matrix for A. In this case C* = C* since C is real. It is easily verified 


1 0 
that C‘AC = . 
0 6 


EXAMPLE 2. If A is already a diagonal matrix, then the diagonalizing matrix C of Theorem 
5.7 either leaves A unchanged or merely rearranges the diagonal entries. 


5.10 Unitary matrices. Orthogonal matrices 


DEFINITION. A square matrix A is called unitary if AA* =I. It is called orthogonal if 
AA‘ =I. 


Note: Every real unitary matrix is orthogonal since A* = A‘. 


Theorem 5.7 tells us that a Hermitian or skew-Hermitian matrix can always be diago- 
nalized by a unitary matrix. A real Hermitian matrix has real eigenvalues and the corre- 
sponding eigenvectors can be taken real. Therefore a real Hermitian matrix can be 
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diagonalized by a real orthogonal matrix. This is not true for real skew-Hermitian matrices. 
(See Exercise 11 in Section 5.11.) 
We also have the following related concepts. 


DEFINITION. A square matrix A with real or complex entries is called symmetric if A = 
A! ; it is called skew-symmetric if A = —A'. 


EXAMPLE 3. If A is real, its adjoint is equal to its transpose, A* = A’. Thus, every real 
Hermitian matrix is symmetric, but a symmetric matrix need not be Hermitian. 


l+i 2 : 1—i 2 l+i 3-i 
EXAMPLE 4, If A = , then A = , A= 
3—i 4i 34+i —4i 2 4i 


1—i 3+i 
and A* = ; 
2 —4i 
1 2 1 2+ 
EXAMPLE 5. Both matrices i and ; are Hermitian. The first is sym- 
—i 3 


metric, the second is not. 


0 —2 i —2 
EXAMPLE 6. Both matrices b i and i 7 are skew-Hermitian. The first is 
i 


skew-symmetric, the second is not. 


EXAMPLE 7. All the diagonal elements of a Hermitian matrix are real. All the diagonal 
elements of a skew-Hermitian matrix are pure imaginary. All the diagonal elements of a 
skew-symmetric matrix are zero. 


EXAMPLE 8. For any square matrix A, the matrix B = $(A + A*) is Hermitian, and the 
matrix C = 3(A — A*) is skew-Hermitian. Their sum is A. Thus, every square matrix 
A can be expressed as asum A = B + C, where Bis Hermitian and C is skew-Hermitian. 
It is an easy exercise to verify that this decomposition is unique. Also every square matrix 
A can be expressed uniquely as the sum of a symmetric matrix, $(A + A‘), and a skew- 
symmetric matrix, (4 — A’). 


EXAMPLE 9. If A is orthogonal we have 1 = det (AA‘) = (det A)(det A’) = (det A)?, so 
det A = +1. 


5.11 Exercises 


1. Determine which of the following matrices are symmetric, skew-symmetric, Hermitian, skew- 
Hermitian. 


012 0 7 2 0 i 2 0 1 2 
(a) /1 031, (li o 3], @l—-~ o 3], @Il-1 0 3 
23 4 a eee a> 3.0 => 3. 26 
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cos @ —sin 6 


2. (a) Verify that the 2 x 2 matrix A = is an orthogonal matrix. 


sin@ cos @ 
(b) Let 7 be the linear transformation with the above matrix A relative to the usual basis 
{i,j}. Prove that T maps each point in the plane with polar coordinates (r, «) onto the point 
with polar coordinates (r, « + 6). Thus, Tis a rotation of the plane about the origin, 6 being 
the angle of rotation. 

3. Let V be real 3-space with the usual basis vectors i, j, k. Prove that each of the following 
matrices is orthogonal and represents the transformation indicated. 


1 0 0 
(a) |0 ] 0 (reflection in the xy-plane). 


0 oO -!1 
1 0 O 
(b) }O —1 0 (reflection through the x-axis). 
0 oO -!1 
—] 0 O 


(c) 0 —-1 0 (reflection through the origin). 


0 0 —1 
1 0 0 
(d) |}0 cos@ —sin 6 (rotation about the x-axis). 


0 sin@ cos @ 
—1] 0 0 
(e) 0 cos@ -—sin 6 (rotation about x-axis followed by reflection in the yz-plane). 


0 sin@ cos @ 


4. A real orthogonal matrix A is called proper if det A = 1, and improper if det A = —1. 
cos @ —sin 6 
(a) If Aisa proper 2 Xx 2 matrix, provethat A = | _ for some 9. This represents 
a rotation through an angle 0. sin@ cos 6 


1 O —1 0 
(b) Prove that R : and ‘ : are improper matrices. The first matrix represents a 


reflection of the xy-plane through the x-axis; the second represents a reflection through the 
y-axis. Find all improper 2 x 2 matrices. 


In each of Exercises 5 through 8, find (a) an orthogonal set of eigenvectors for A, and (b) 
a unitary matrix C such that C~1AC is a diagonal matrix. 


9 12 0: =? 
5. A= 6. A= 
12 16 2 O 
3 O 1 3 4 
7.A4=13 —2 —-1|. 8. A=|13 1 0 


0-1 1 40 1 
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9, Determine which of the following matrices are unitary, and which are orthogonal (a, 5, 6 real). 


; cos@ 0 sin é@ kV2 -1V3 iv6 
ett : a 
(a) i ‘ » (db) 0 1 Of, ©) 0 V3 1V6 
e* z = 2 

—sin@ O cosé@ LV2)4V30 2 V6 


10. The special theory of relativity makes use of the equations 


, 


‘sa(x—vt), ysy, =z, t=alt — vac’). 


Here v is the velocity of a moving object, c the speed of light, and a = c/Vc? —v?. The 
linear transformation which maps (x, y, z, t)onto (x’, y’, z’, t’)iscalleda Lorentz transformation. 
(a) Let (x1, X2, X3, %4) = (x, y, Z, ict) and (x), X, X3, %4) = (x’, y’, 2’, ict’). Show that the 
four equations can be written as one matrix equation, 


a 00 -iavfc 
7 oe 0 1 0 0 
[x1 Xo, X%y, Xq4) = [%1, Xo, X3, X4] . ‘ 
iavlce 0 O a 


(b) Prove that the 4 x 4 matrix in (a) is orthogonal but not unitary. 


11. Let a be a nonzero real number and let A be the skew-symmetric matrix A = : ‘ 
—a 
(a) Find an orthonormal set of eigenvectors for A. 
(b) Find a unitary matrix C such that C~'AC is a diagonal matrix. 
(c) Prove that there is no real orthogonal matrix C such that C~'AC is a diagonal matrix. 
12. If the eigenvalues of a Hermitian or skew-Hermitian matrix A are all equal to c, prove that 
A=cl. 
13. If A is a real skew-symmetric matrix, prove that both J — A and J + A are nonsingular and 
that (I — A)U + A) is orthogonal. 
14. For each of the following statements about n x n matrices, give a proof or exhibit a counter 
example. 
(a) If A and Bare unitary, then A + B is unitary. 
(b) If A and B are unitary, then AB is unitary. 
(c) If A and AB are unitary, then B is unitary. 
(d) If A and Bare unitary, then A + Bis not unitary. 


5.12 Quadratic forms 


Let V be a real Euclidean space and let T: V > V be a symmetric operator. This means 
that T can be shifted from one factor of an inner product to the other. 


(T(x), y) =(x,TQ)) forall x and yin V. 
Given 7, we define a real-valued function Q on V by the equation 


Q(x) = (7(x), x). 
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The function Q is called the quadratic form associated with T. The term “quadratic’”’ is 
suggested by the following theorem which shows that in the finite-dimensional case Q(x) 
is a quadratic polynomial in the components of x. 


THEOREM 5.8. Let (e,,...,¢,) be an orthonormal basis for a real Euclidean space V. 
Let T: V— V be a symmetric transformation, and let A = (a;;) be the matrix of T relative 
to this basis. Then the quadratic form Q(x) = (T(x), x) is related to A as follows: 


(5.7) O(x) = > Ya A; ;X;X; if x= > X,e;- 


Proof. By linearity we have T(x) = > x;T(e,). Therefore 


Q(x) = (ZiT). 3x24) = > ¥ xx/(T(e). é;). 


This proves (5.7) since a;; = a;; = (T(e;), e;). 


The sum appearing in (5.7) is meaningful even if the matrix A is not symmetric. 


DEFINITION. Let V be any real Euclidean space with an orthonormal basis (e,,..., @,), and 
let A = (a;;) by any n X n matrix of scalars. The scalar-valued function Q defined at each 
element x = > x,e; of V by the double sum 


(5.8) Ox) = ¥ Sax, 


iM 


is called the quadratic form associated with A. 


If A is a diagonal matrix, then a,;; = 0 if i ¥ j' so the sum in (5.8) contains only squared 
terms and can be written more simply as 


Ox) = Yay? 


In this case the quadratic form is called a diagonal form. 
The double sum appearing in (5.8) can also be expressed as a product of three matrices. 


THEOREM 5.9. Let X = [x,,...,X,] be a 1 Xn row matrix, and let A = (a,;) be an 
n xX nmatrix. Then XAX‘ is al xX 1 matrix with entry 


(5.9) > > 4i5%iX;- 


Proof. The product XA isal x nmatrix, XA = [y,,...,y,], where entry y, is the dot 
product of X with the jth column of A, 


n 
t=1 
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Therefore the product XAX‘ is a 1 x 1 matrix whose single entry is the dot product 


n n n n 
2% =< Z (3 xa) x5 = > Dd aijxiX;- 
I= I= 


i=1 t=1 j=1 


Note: It is customary to identify the 1 x 1 matrix XAX‘ with the sum in (5.9) and to 
call the product XAX‘a quadratic form. Equation (5.8) is written more simply as follows: 


O(x) = XAX*. 


EXAMPLE |. Let A = ; , X = [x,, x.]. Then we have 


XA = [x1, nl y a [x4 =, 3Xe, aoe. faa 5X2], 


and hence 
xX) 


XAX' = [x — 3xg, —X, + sel 


= x7 — 3x0x, — XX. + SXF. 
Xe 


EXAMPLE 2. Let B = i , X = [x,, X,]. Then we have 


1 —2||x 
XBX' = [x,, nl ; al| ‘ = x? — 2x.x, — 2x1x, + 5x3. 


In both Examples | and 2 the two mixed product terms add up to —4x,x, so XAX! = 
XBX'. These examples show that different matrices can lead to the same quadratic form. 
Note that one of these matrices is symmetric. This illustrates the next theorem. 


THEOREM 5.10. For any n X n matrix A and any | X n row matrix X we have XAX' = 
XBX' where B is the symmetric matrix B = 3(A + A’). 


Proof. Since XAX‘ is a 1 x 1 matrix it is equal to its transpose, XAX! = (XAX")'. 
But the transpose of a product is the product of transposes in reversed order, so we have 
(XAX')' = XA'X'. Therefore XAX' = $XAX' + $XA'X' = XBX'. 


5.13 Reduction of a real quadratic form to a diagonal form 


A real symmetric matrix A is Hermitian. Therefore, by Theorem 5.7 it is similar to the 
diagonal matrix A = diag (A,,..., 4,) of its eigenvalues. Moreover, we have A = C'AC, 
where C is an orthogonal matrix. Now we show that C can be used to convert the quadratic 
form XAX' to a diagonal form. 
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THEOREM 5.11. Let XAX* be the quadratic form associated with a real symmetric matrix 
A, and let C be an orthogonal matrix that converts A to a diagonal matrix A = C'AC. 
Then we have 


XAX= YAY'= > Avi, 


where Y = [y1,..., Yn] is the row matrix Y = XC, andd,,...,4, are the eigenvalues of A. 


Proof. Since C is orthogonal we have C-}=C*. Therefore the equation Y = XC 
implies X = YC‘, and we obtain 


XAX' = (YC‘)A(YC**t = Y(C'AC)Y' = YAY'*. 


Note: Theorem 5.11 is described by saying that the linear transformation Y = XC 
reduces the quadratic form XAX‘ to a diagonal form YAY*. 


EXAMPLE |. The quadratic form belonging to the identity matrix is 
XIX! = Y x; = XI’, 
i=1 


the square of the length of the vector ¥ = (x,,...,x,). Alinear transformation Y = XC, 
where C is an orthogonal matrix, gives a new quadratic form YAY! with A = CIC’ = 
CC' =I]. Since XIX‘ = YIY' we have ||X||? = || Y||?, so Y has the same length as X. A 
linear transformation which preserves the length of each vector is called an isometry. 
These transformations are discussed in more detail in Section 5.19. 


EXAMPLE 2. Determine an orthogonal matrix C which reduces the quadratic form Q(x) = 
2x2 + 4x4X_ + 5x§ to a diagonal form. 


Z: 2 
Solution. We write Q(x) = XAX', where A = Ps | This symmetric matrix was 


diagonalized in Example 1 following Theorem 5.7. It has the eigenvalues 4, = 1, A, = 6, 
and an orthonormal set of eigenvectors u,, u,, where u, = ¢(2, —1), up = t(1,2), t= 


- 2 1 
/5. An orthogonal diagonalizing matrix is C= | | The corresponding 
diagonal form is - 


YAY' = Ay? + dey? = yi + 6y?. 


The result of Example 2 has a simple geometric interpretation, illustrated in Figure 5.1. 
The linear transformation Y = XC can be regarded as a rotation which maps the basis @, 
j onto the new basis u,, u,. A point with coordinates (x,, x.) relative to the first basis has 
new coordinates (),, y.) relative to the second basis. Since XAX'= YAY‘, the set of 
points (x,, x2) satisfying the equation XAX* = c for some c is identical with the set of 
points ()1, 2) satisfying YAY'‘=c. The second equation, written as ? + 613 =, Is 
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y2 : 
( (xX, X2) relative to basis i, j 
e 


| (1, ¥2) relative to basis u,, up 


Ficure 5.1 Rotation of axes by an orthogonal matrix. The ellipse has Cartesian equation 
XAX* = 9 in the x,x.-system, and equation YAY’ = 9 in the y,ye-system. 


the Cartesian equation of an ellipse if c > 0. Therefore the equation XYAX* = c, written 
as 2x? + 4x,x, + 5x2 = c, represents the same ellipse in the original coordinate system. 
Figure 5.1 shows the ellipse corresponding to c = 9. 


5.14 Applications to analytic geometry 


The reduction of a quadratic form to a diagonal form can be used to identify the set of 
all points (x, y) in the plane which satisfy a Cartesian equation of the form 


(5.10) ax? + bxy+cy?>+dx+ey+f=0. 


We shall find that this set is always a conic section, that is, an ellipse, hyperbola, parabola, 
‘or one of the degenerate cases (the empty set, a single point, or one or two straight lines). 
The type of conic is governed by the second-degree terms, that is, by the quadratic form 
ax® + bxy + cy*. To conform with the notation used earlier, we write x, for x, x, for y, 
and express this quadratic form as a matrix product, 


XAX* = ax? + bx x2 + cx?, 


a b/2 
where X = [x,, X,] and A = 
b/2 ¢ 
a diagonal form A,y? + A,y?, where A,, A, are the eigenvalues of A. An orthonormal set 
of eigenvectors u, , v, determines a new set of coordinate axes, relative to which the Cartesian 
equation (5.10) becomes 


| . By arotation Y = XC we reduce this form to 


(5.11) Ayitdystdytey,.tf=0, 


with new coefficients d’ and e’ in the linear terms. In this equation there is no mixed product 
term y V2, so the type of conic is easily identified by examining the eigenvalues A, and A,. 
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If the conic is not degenerate, Equation (5.11) represents an ellipse if A,, 4, have the same 
sign, a hyperbola if 4,, A, have opposite signs, and a parabola if either A, or A, is zero. The 
three cases correspond to AA, > 0, 4,A, <0, and AA, = 0. We illustrate with some 
specific examples. 


EXAMPLE |. 2x2 + 4xy + Sy? + 4x + 13y — 4 =0. We rewrite this as 
(5.12) 2x2 + 4x x5 + 5x% + 4x, + 13x,-—}4=0. 


The quadratic form 2x? + 4x,x, + 5x? is the one treated in Example 2 of the foregoing 
section. Its matrix has eigenvalues A, = 1, A, = 6, and an orthonormal set of eigenvectors 


u, = t(2, —1), u, = ¢t(1, 2), where t = 1/J5. An orthogonal diagonalizing matrix is 
2 1 
C= | ; | This reduces the quadratic part of (5.12) to the form y? + 6y3. To 


determine the effect on the linear part we write the equation of rotation Y = XC in the 
form X = YC* and obtain 


1 2 —-1 
[x1, x2] = 5 [yi » ral af x= ‘ ze 2y1 + 2), ee = ‘ (—Yi + 2y2). 
Therefore the linear part 4x, + 13x, is transformed to 


Qy, +) + a(~ Vi t+ 2yo) = —J/5 y, + 64/5 yo. 


v5 Vv 


The transformed Cartesian equation becomes 


yi + 6y3 — /5y, + 6/5 y2 —$ =0. 


By completing the squares in y, and y, we rewrite this as follows: 
— 5) + 6(y2 + WV)? = 9, 


This is the equation of an ellipse with its center at the point (AV 5, ay 5) in the y,ye- 
system. The positive directions of the y, and y, axes are determined by the eigenvectors wu, 
and u,, as indicated in Figure 5.2. 

We can simplify the equation further by writing 


SANS. 2, = yo t WS. 


Geometrically, this is the same as introducing a new system of coordinate axes parallel to 
the y,y. axes but with the new origin at the center of the ellipse. In the z,z.-system the 
equation of the ellipse is simply 

2 
ie ere 


z+6z2=9, or 5 3/2 


The ellipse and all three coordinate systems are shown in Figure 5.2. 
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FiGurE 5.2 Rotation and translation of coordinate axes. The rotation Y = XC 
is followed by the translation z; = y, — ie, 5,2, = yo + 1,/ 5. 


EXAMPLE 2. 2x? — 4xy — y? — 4x + 10y — 13 =0. We rewrite this as 
2x} — 4x4x_, — x2 — 4x, + 10x, — 13 = 0. 


—2 


The quadratic part is XAX', where A = . This matrix has the eigenvalues 


A, = 3, A, = —2. An orthonormal set of eigenvectors is u, = ¢(2, —1), u, = t(1, 2), 


= 2: 1 
where t = 1/5. An orthogonal diagonalizing matrix is C = | | . The equation 
of rotation ¥ = YC‘ gves us —1 2 


1 1 
x, = Wins + yo), X= —=(—y, + 2yz). 


J5 


Therefore the transformed equation becomes 


4 10 
3y} = 2y3 am Yh. + Yo) + fas! + 2y.) —-13 =0, 
or 
18 16 
3y, — 2yg —- zy t+ = y, — 13 = 0. 


af 5 


By completing the squares in y, and y, we obtain the equation 


3(y, — 3V5)* — 2(y, — 4v'5)? = 12, 
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Xo Vi 
yea 
Us u, 
x 
(a) Hyperbola: 3z? — 223 = 12 (b) Parabola: y? + y, = 0 


FiGuRE 5.3. The curves in Examples 2 and 3. 


which represents a hyperbola with its center at (3,/ 5, 4,/ 5) in the y,y.-system. The trans- 
lation z; = y, — 3/5, 2 = yo 44/5 simplifies this equation further to 


2 2 
gS ae. 6k ee 
4 6 


The hyperbola is shown in Figure 5.3(a). The eigenvectors u, and u, determine the direc- 
tions of the positive y, and y, axes. 


EXAMPLE 3, 9x? + 24xy + l6y? — 20x + IS5y =0. We rewrite this as 


Ox? + 24x1x_ + 16x — 20x, + 15x, = 0. 


12 
A, = 0. An orthonormal set of eigenvectors is u, = $(3, 4), u, = $(—4, 3). An orthog- 


The symmetric matrix for the quadratic partis A = . Its eigenvalues are A, = 25, 


3 —4 
onal diagonalizing matrix is C = H 7 . The equation of rotation ¥ = YC" gives us 


x, = 3(3y1 — 4y2), X2 = (4, + 3y2). 
Therefore the transformed Cartesian equation becomes 
25yi — *2(3y, — 4y2) + 78(4y1 + 3y2) = 0. 


This simplifies to y? + y, = 0, the equation of a parabola with its vertex at the origin. The 
parabola is shown in Figure 5.3(b). 
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EXAMPLE 4. Degenerate cases. A knowledge of the eigenvalues alone does not reveal 
whether the Cartesian equation represents a degenerate conic section. For example, the 
three equations x? + 2y>=1, x? +4+2y?=0, and x? + 2y? = —1 all have the same 
eigenvalues; the first represents a nondegenerate ellipse, the second is satisfied only by 
(x, y) = (0, 0), and the third represents the empty set. The last two can be regarded as 
degenerate cases of the ellipse. 

The graph of the equation y? = 0 is the x-axis. The equation y? — 1 = 0 represents the 
two parallel lines y = 1 and y = —1. These can be regarded as degenerate cases of the 
parabola. The equation x* — 4y? = 0 represents two intersecting lines since it is satisfied 
if either x — 2y = 0 or x + 2y =0. This can be regarded as a degenerate case of the 
hyperbola. 

However, if the Cartesian equation ax* + bxy + cy? + dx +ey+/f=0 represents 
a nondegenerate conic section, then the type of conic can be determined quite easily. The 
characteristic polynomial of the matrix of the quadratic form ax? + bxy + cy? Is 


Po i | Oe | Pea IO 
ln _\- (a+ OA + (ac ~ 36) = (A A)A- A). 


Therefore the product of the eigenvalues is 
Ayo = ac— 15? = 4(4ac a b?) . 


Since the type of conic is determined by the algebraic sign of the product A,A,, we see that 
the conic is an ellipse, hyperbola, or parabola, according as 4ac — b? is positive, negative, or 
zero. The number 4ac — b? is called the discriminant of the quadratic form ax? + bxy + 
cy?. In Examples 1, 2 and 3 the discriminant has the values 34, —24, and 0, respectively. 


5.15 Exercises 


In each of Exercises 1 through 7, find (a) a symmetric matrix A for the quadratic form; (b) the 
eigenvalues of A; (c) an orthonormal set of eigenvectors; (d) an orthogonal diagonalizing matrix C. 


1. 4x? + 4xx_ + x5. 5. x8 + XyXq + 1X3 + XoXo. 
2. X4Xe. 6. 2x? + 4x1xg + xh — x8. 
3. x? + 2xyx. — xe. 7. 3x2 + 4x,xXq + 8xyx3 + 4x—x3 + 3x3. 


4. 34x? — 24x ,xq + 41x5. 


In each of Exercises 8 through 18, identify and make a sketch of the conic section represented by 
the Cartesian equation. 


8. y®? — 2xy + 2x* —5 =0. 14. 5x? + Oxy +5y? —2 =0. 

9. y2 — 2xy + 5x =0. 15. x2 + 2xy + y? —2x +2y +3 =0. 
10. y? — 2xy + x? — Sx =0. 16. 2x2 + 4xy + Sy? —-2x —y—4=0. 
11. 5x? — 4xy + 2y? —6 =0. 17. x2 + 4xy — 2y? — 12 =0. 

12. 19x2 + 4xy + 16y? — 212x + 104y = 356. 

13. 9x? 4+ 24xy + 16y? — 52x + 1l4y = 6. 18. xy +y —2x —2=0. 


19. For what value (or values) of c will the graph of the Cartesian equation 2xy — 4x + 7y +c = 
0 be a pair of lines? 
20. If the equation ax? + bxy + cy* = 1 represents an ellipse, prove that the area of the region it 


bounds is 27/V4ac — b®, This gives a geometric meaning to the discriminant 4ac — b?. 
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* 5.16, Eigenvalues of a symmetric transformation obtained as values of its quadratic form 


Now we drop the requirement that V be finite-dimensional and we find a relation between 
the eigenvalues of a symmetric operator and its quadratic form. 
Suppose x is an eigenvector with norm | belonging to an eigenvalue A. Then T(x) = Ax 


SO we have 
(5.13) O(x) = (T(x), x) = (Ax, x) = A(x, x) =A, 


since (x, x) = 1. The set of all x in V satisfying (x, x) = 1 1s called the unit sphere in V. 
Equation (5.13) proves the following theorem. 


THEOREM 5.12. Let T: V— V be asymmetric transformation on a real Euclidean space V, 
and let Q(x) = (T(x), x). Then the eigenvalues of T (if any exist) are to be found among the 
values that Q takes on the unit sphere in V. 


EXAMPLE. Let V = V,(R) with the usual basis (i,j) and the usual dot product as inner 


0 
product. Let T be the symmetric transformation with matrix A = ( Then the 
quadratic form of T is given by 0 8 


2 
2 2 
> Gj,X:X; = 4x} + 8X5. 
1 j=1 


Mr 


Q(x) = 


t 


The eigenvalues of T are A, = 4, A, = 8. It is easy to see that these eigenvalues are, 
respectively, the minimum and maximum values which Q takes on the unit circle x? + 
x§ = 1. In fact, on this circle we have 


Q(x) = 4(xi + x5) + 4x3 = 4+ 4x5, where —1<x,<1. 


This has its smallest value, 4, when x, = 0 and its largest value, 8, when x, = +1. 

Figure 5.4 shows the unit circle and two ellipses. The inner ellipse has the Cartesian 
equation 4x? + 8x2 = 4. It consists of all points x = (x,, x2) in the plane satisfying 
Q(x) =4. The outer ellipse has Cartesian equation 4x? + 8x? = 8 and consists of all 
points satisfying Q(x) = 8. The points (+1, 0) where the inner ellipse touches the unit 
circle are eigenvectors belonging to the eigenvalue 4. The points (0, +1) on the outer 
ellipse are eigenvectors belonging to the eigenvalue 8. 


The foregoing example illustrates extremal properties of eigenvalues which hold more 
generally. In the next section we will prove that the smallest and largest eigenvalues (if 
they exist) are always the minimum and maximum values which Q takes on the unit sphere. 
Our discussion of these extremal properties will make use of the following theorem on 
quadratic forms. It should be noted that this theorem does not require that V be finite 
dimensional. 


+ Starred sections can be omitted or postponed without loss in continuity. 
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Eigenvector belonging to 8 


Q(x) = 8 on this ellipse 


x 
Eigenvector belonging to 4 
Q(x) = 40n this ellipse 


FIGURE 5.4 Geometric relation between the eigenvalues of T and the values of Q 
on the unit sphere, illustrated with a two-dimensional example. 


THEOREM 5.13. Let T: V— V be a symmetric transformation on a real Euclidean space V 
with quadratic form Q(x) = (T(x), x). Assume that Q does not change sign on V. Then if 
Q(x) = 0 for some x in V we also have T(x) = O. In other words, if Q does not change sign, 
then Q vanishes only on the null space of T. 


Proof. Assume Q(x) = 0 for some x in V and let y be any element in V. Choose any 
real ¢t and consider Q(x + ty). Using linearity of 7, linearity of the inner product, and 
symmetry of T, we have 


O(x + ty) = (T(x + ty), x + ty) = (T(x) + (TQ), x + ty) 
= (T(x), x) + t(T(x), y) + (Ty), x) + (70), Y) 
= Q(x) + 2t(T(x), y) + Q(y) = at + bt?, 


where a = 2(7(x), y) and b = Q(y). If Q is nonnegative on V we have the inequality 
at + bt? >0 for all real ¢. 


In other words, the quadratic polynomial p(t) = at + bt? has its minimum at t= 0. 
Hence p'(0) = 0. But p’(0) = a = 2(7(x), y), so (T(x), y) = 0. Since y was arbitrary, 
we can in particular take y = T(x), getting (7(x), T(x)) = 0. This proves that T(x) = O. 

If Q is nonpositive on V we get p(t) = at + bt? < 0 for all t, so p has its maximum at 
t = 0, and hence p’(0) = 0 as before. 


+ 5.17 Extremal properties of eigenvalues of a symmetric transformation 
Now we shall prove that the extreme values of a quadratic form on the unit sphere are 


eigenvalues. 


THEOREM 5.14. Let T: V—> V be a symmetric linear transformation on a real Euclidean 
space V, and let Q(x) = (T(x), x). Among all values that Q takes on the unit sphere, assume 
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there is an extremum{ (maximum or minimum) at a point u with (u, u) = 1. Thenuisaneigen- 
vector for T; the corresponding eigenvalue is Q(u), the extreme value of Q on the unit sphere. 


Proof. Assume Q has a minimum at u. Then we have 
(5.14) O(x) > Ou) for all x with (x, x) = 1. 


Let A = Q(u). If (x, x) = 1 we have Q(u) = A(x, x) = (Ax, x) so inequality (5.14) can be 
written as 


(5.15) (T(x), x) > (Ax, x) 


provided (x, x) = 1. Now we prove that (5.15) is valid for all x in V. Suppose ||x|| = a. 
Then x = ay, where ||y|| = 1. Hence 


(T(x), x) = (Tay), ay) = a@?(TQ), yy) = and — (Ax, x) = a®(Ay, y). 


But (T(y), y) > (Ay, y) since (y, y) = 1. Multiplying both members of this inequality by 
a* we get (5.15) for x = ay. 

Since (T(x), x) — (Ax, x) = (T(x) — 4x, x), we can rewrite inequality (5.15) in the form 
(T(x) — Ax, x) > 0, or 


(5.16) (S(x), x) > 0, where S=T—AI. 


When x = u we have equality in (5.14) and hence also in (5.16). The linear transformation 
S is symmetric. Inequality (5.16) states that the quadratic form Q, given by Q,(x) = 
(S(x), x) is nonnegative on V. When x = u we have Q,(u) = 0. Therefore, by Theorem 
5.13 we must have S(u) = O. In other words, J(u) = Au, so wis an eigenvector for T, and 
A = Q(u) is the corresponding eigenvalue. This completes the proof if Q has a minimum 
at u. 

If there is a maximum at u all the inequalities in the foregoing proof are reversed and we 
apply Theorem 5.13 to the nonpositive quadratic form Q,. 


% 5.18 The finite-dimensional case 


Suppose now that dim V =n. Then T has x real eigenvalues which can be arranged in 
increasing order, say 
Ay Sag St SA. 


According to Theorem 5.14, the smallest eigenvalue 4, is the minimum of Q on the unit 
sphere, and the largest eigenvalue is the maximum of Q on the unit sphere. Now we shall 
show that the intermediate eigenvalues also occur as extreme values of Q, restricted to 
certain subsets of the unit sphere. 


+ If Vis infinite-dimensional, the quadratic form Q need not have an extremum on the unit sphere. This will 
be the case when 7 has no eigenvalues. In the finite-dimensional case, Q always has a maximum and a 
minimum somewhere on the unit sphere. This follows as a consequence of a more general theorem on 
extreme values of continuous functions. For a special case of this theorem see Section 9.16. 
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Let u, be an eigenvector on the unit sphere which minimizes Q. Then A, = Q(u,). If A 
is an eigenvalue different from A, any eigenvector belonging to A must be orthogonal to u,. 
Therefore it is natural to search for such an eigenvector on the orthogonal complement of 
the subspace spanned by u,. 

Let S be the subspace spanned by u,. The orthogonal complement S~* consists of all 
elements in V orthogonal to u,. In particular, S‘ contains all eigenvectors belonging to 
eigenvalues A  A,. It is easily verified that dim S' = n — 1 and that T maps S* into 
itself.f Let S,_, denote the unit sphere in the (m — 1)-dimensional subspace S+. (The 
unit sphere S,,_, is a subset of the unit sphere in V.) Applying Theorem 5.14 to the subspace 
S+ we find that A, = Q(u,.), where u, is a point which minimizes Q on S,,_,. 

The next eigenvector A, can be obtained in a similar way as the minimum value of Q on 
the unit sphere S,,_, in the (n — 2)-dimensional space consisting of those elements orthog- 
onal to both u, and u,. Continuing in this manner we find that each eigenvalue 4, is the 
minimum value which @Q takes on a unit sphere S,,_;,,, in a subspace of dimensionn —k+ 1. 
The largest of these minima, A, , is also the maximum value which Q takes on each of the 
spheres S,,_,,,. The corresponding set of eigenvectors u,,...,u, form an orthonormal 
basis for V. 


5.19 Unitary transformations 


We conclude this chapter with a brief discussion of another important class of trans- 
formations known as unitary transformations. In the finite-dimensional case they are 
represented by unitary matrices. 


DEFINITION. Let E be a Euclidean space and V a subspace of E. A linear transformation 
T: V — Eis called unitary on V if we have 


(5.17) (T(x), T(y)) = (x,y) ~— for all x and y in V. 


When E is a real Euclidean space a unitary transformation is also called an orthogonal 
transformation. 


Equation (5.17) is described by saying that T preserves inner products. Therefore it is 
natural to expect that T also preserves orthogonality and norms, since these are derived 
from the inner product. 


THEOREM 5.15. Jf T: V — Eis a unitary transformation on V, then for all x and y in V we 
have: 

(a) (x, y) = 0 implies (T(x), T(y)) = 0 (T preserves orthogonality). 

(b) || 7(x)|| = |x] (T preserves norms). 

(c) ||7(~) — Tiy)|| = |lx — yl (T preserves distances). 

(d) 7 is invertible, and T™ is unitary on T(V). 


} This was done in the proof of Theorem 5.4, Section 5.6. 
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Proof. Part (a) follows at once from Equation (5.17). Part (b) follows by taking x = y 
in (5.17). Part (c) follows from (b) because T(x) — T(y) = T(x — y). 

To prove (d) we use (b) which shows that T(x) = O implies x = O, so T is invertible. 
If x € T(V) and y € T(V) we can write x = T(u), y = T(v), so we have 


(T(x), Ty) = G, v) = (TH), Tr) = (*, y). 
Therefore T~! is unitary on 7(V). 


Regarding eigenvalues and eigenvectors we have the following theorem. 


THEOREM 5.16. Let T: V— E be a unitary transformation on V. 
(a) If T has an eigenvalue 4, then |A| = 1. 


(b) If x and y are eigenvectors belonging to distinct eigenvalues 4 and wu, then x and y are 
orthogonal. 

(c) If V = E and dim V = n, and if V is a complex space, then there exist eigenvectors 
U,,...,U, of T which form an orthonormal basis for V. The matrix of T relative to 
this basis is the diagonal matrix A = diag (A,,...,4,), where 4, is the eigenvalue 


belonging to u,. | 


Proof. To prove (a), let x be an eigenvector belonging to A. Then x ¥ Oand T(x) = 
Taking y = x in Equation (5.17) we get 


(Ax, Ax) =(x,x) or = AA(x, x) = (x, x). 


Since (x, x) > 0 and AA = |A|?, this implies |A| = 1. 
To prove (b), write T(x) = Ax, T(y) = wy and compute the inner product (7(x), T())) 
in two ways. We have 
(T(x), T(Y)) = (x, y) 


since J is unitary. We also have 


(T(x), TY) = (Ax, wy) = AM(x, y) 


since x and y are eigenvectors. Therefore Af(x, y) = (x, y), so (x, y) = 0 unless Afi = 1. 
But 4A = 1 by (a), so if we had A@ = 1 we would also have AA = Ag, A = fi, A = mw, which 
contradicts the assumption that A and w are distinct. Therefore A@ # 1 and (x,y) = 0. 

Part (c) is proved by induction on 7 in much the same way that we proved Theorem 5.4, 
the corresponding result for Hermitian operators. The only change required is in that part 
of the proof which shows that T maps S“ into itself, where 


= {x|xeEV, (x,u,) = 0}. 
Here uw, is an eigenvector of T with eigenvalue A,. From the equation T(u,) = A,u, we tind 


y= Ay*T(uy) = A,T(u;) 
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since A,A, = |A,|? = 1. Now choose any x in S+ and note that 
(T(x), 41) = (T(x), A,T(u,)) = A,(7(x), T(u1)) = A(x, uy) = 0. 


Hence 7(x) € S+ if x € S', so T maps S~ into itself. The rest of the proof is identical with 
that of Theorem 5.4, so we shall not repeat the details. 


The next two theorems describe properties of unitary transformations on a finite- 
dimensional space. We give only a brief outline of the proofs. 


THEOREM 5.17. Assume dim V =n and let E = (e,...,@,) be a fixed basis for V. 
Then a linear transformation T: V — V is unitary if and only if 


(5.18) (T(e,), T(e)) = (e:,¢;) for all i andj. 


In particular, if E is orthonormal then T is unitary if and only if T maps E onto an orthonormal 
basis. 


Sketch of proof. Write x = > x,e;, y = > y;e;. Then we have 


(x,y) = (dx > y.e1) = > 
1 jal = 


n 
> X:Vilei> es), 
j=l 


x“. 


and 


(TO, TO) = (Sx.7@).Zy,T(e)) = ¥ YxH(TE). Te). 


t=1 j=1 


Now compare (x, y) with (T(x), T(y)). 


THEOREM 5.18. Assume dim V =n and let (e,,...,e,) be an orthonormal basis for V. 
Let A = (a;;) be the matrix representation of a linear transformation T: V — V relative to 
this basis. Then T is unitary if and only if A is unitary, that is, if and only if 
(5.19) A*A =I, 


Sketch of proof. Since (e;, e;) is the ij-entry of the identity matrix, Equation (5.19) 
implies 


(5.20) (2553) = > Gdns = D Api; - 
k=1 k=1 


Since A is the matrix of T we have T(e,) = >%_, aye,, T(e;) = >”_1 4,j€, 5 $0 


k=1 r=1 


(T(e;), T(e;)) = (Sane > a,¢, = > > Api A, (Cp, Cp) = D> Ueidy 5 « 
= r= k=1 


Now compare this with (5.20) and use Theorem 5.17. 
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THEOREM 5.19. Every unitary matrix A has the following properties: 
(a) A is nonsingular and A“! = A*. 

(b) Each of At, A, and A* is a unitary matrix. 

(c) The eigenvalues of A are complex numbers of absolute value |. 
(d) |det A] = 1; if A is real, then det A= +1. 


The proof of Theorem 5.19 is left as an exercise for the reader. 


5.20 Exercises 


l. 


14. 


(a) Let T: V + V be the transformation given by T(x) = cx, where c is a fixed scalar. Prove 
that Tis unitary if and only if |c| = 1. 

(b) If V is one-dimensional, prove that the only unitary transformations on V are those des- 
cribed in (a). In particular, if Vis a real one-dimensional space, there are only two orthogonal 
transformations, T(x) = x and T(x) = —x. 


. Prove each of the following statements about a real orthogonal n x n matrix A. 


(a) If A is a real eigenvalue of A, then A = 1loréA= —1. 

(b) If 2 is a complex eigenvalue of A, then the complex conjugate 4 is also an eigenvalue of A. 
In other words, the nonreal eigenvalues of A occur in conjugate pairs. 

(c) If nis odd, then A has at least one real eigenvalue. 


. Let V be a real Euclidean space of dimension n. An orthogonal transformation T: V > V 


with determinant 1 is called a rotation. If n is odd, prove that | is an eigenvalue for 7. This 
shows that every rotation of an odd-dimensional space has a fixed axis. [Hint: Use Exercise 2.] 


. Given a real orthogonal matrix A with —1 as an eigenvalue of multiplicity k. Prove that det A = 


(—1)*. 


. If Tis linear and norm-preserving, prove that 7 is unitary. 
. If T: V > Vis both unitary and Hermitian, prove that T? = I. 
. Let (e,,..., en) and (u,,..., Up) be two orthonormal bases for a Euclidean space V. Prove 


that there is a unitary transformation T which maps one of these bases onto the other. 


. Find a real a such that the following matrix is unitary: 
a $i 4a(2i — 1) 
ia 401 +4) gal —2) 
a -b MQ -/ 


. If A is a skew-Hermitian matrix, prove that both J — A and J + A are nonsingular and that 


(I — A)U + A) is unitary. 


. If A is a unitary matrix and if J + A is nonsingular, prove that (J — A)(J + A) is skew- 


Hermitian. 


. If A is Hermitian, prove that A — iJ is nonsingular and that (A — iI)"‘(A + iJ) is unitary. 
. Prove that any unitary matrix can be diagonalized by a unitary matrix. 


. A square matrix is called normal if AA* = A*A. Determine which of the following types of 
matrices are normal. 
(a) Hermitian matrices. (d) Skew-symmetric matrices. 
(b) Skew-Hermitian matrices. (e) Unitary matrices. 
(c) Symmetric matrices. (f) Orthogonal matrices. 


If A is anormal matrix (44* = A*A) and if Uis a unitary matrix, prove that U* AU is normal. 
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LINEAR DIFFERENTIAL EQUATIONS 


6.1 Historical introduction 


The history of differential equations began in the 17th century when Newton, Leibniz, and 
the Bernoullis solved some simple differential equations of the first and second order arising 
from problems in geometry and mechanics. These early discoveries, beginning about 1690, 
seemed to suggest that the solutions of all differential equations based on geometric and 
physical problems could be expressed in terms of the familiar elementary functions of 
calculus. Therefore, much of the early work was aimed at developing ingenious techniques 
for solving differential equations by elementary means, that is to say, by addition, sub- 
traction, multiplication, division, composition, and integration, applied only a finite 
number of times to the familiar functions of calculus. 

Special methods such as separation of variables and the use of integrating factors were 
devised more or less haphazardly before the end of the 17th century. During the 18th 
century, more systematic procedures were developed, primarily by Euler, Lagrange, and 
Laplace. It soon became apparent that relatively few differential equations could be solved 
by elementary means. Little by little, mathematicians began to realize that it was hopeless 
to try to discover methods for solving all differential equations. Instead, they found it more 
fruitful to ask whether or not a given differential equation has any solution at all and, 
when it has, to try to deduce properties of the solution from the differential equation itself. 
Within this framework, mathematicians began to think of differential equations as new 
sources of functions. 

An important phase in the theory developed early in the 19th century, paralleling the 
general trend toward a more rigorous approach to the calculus. In the 1820’s, Cauchy 
obtained the first “‘existence theorem” for differential equations. He proved that every 
first-order equation of the form 

y = f(x, y) 


has a solution whenever the right member, f(x, y), satisfies certain general conditions. 
One important example is the Ricatti equation 


y’ = P(x)y? + O(x)y + ROX), 


where P, Q, and R are given functions. Cauchy’s work implies the existence of a solution 
of the Ricatti equation in any open interval (—r, r) about the origin, provided P, Q, and 
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R have power-series expansions in (—r,r). In 1841 Joseph Liouville (1809-1882) showed 
that in some cases this solution cannot be obtained by elementary means. 

Experience has shown that it is difficult to obtain results of much generality about 
solutions of differential equations, except for a few types. Among these are the so-called 
linear differential equations which occur in a great variety of scientific problems. Some 
simple types were discussed in Volume I—linear equations of first order and linear equations 
of second order with constant coefficients. The next section gives a review of the principal 
results concerning these equations. 


6.2 Review of results concerning linear equations of first and second orders 


A linear differential equation of first order is one of the form 
(6.1) y + PXy = QQ), 


where P and Q are given functions. In Volume I we proved an existence-uniqueness 
theorem for this equation (Theorem 8.3) which we restate here. 


THEOREM 6.1. Assume P and Q are continuous on an open interval J. Choose any point a 
in J and let b be any real number. Then there is one and only one function y = f(x) which 
satisfies the differential equation (6.1) and the initial condition f(a) = b. This function is 
given by the explicit formula 


(6.2) iOS bea pee [ O(t)e4” dt, 


where A(x) = J* P(t) dt. 


Linear equations of second order are those of the form 
Po(x)y" + Pa(x)y’ + Pa(x)y = R(x). 


If the coefficients Py, P,, P, and the right-hand member R are continuous on some interval 
J, and if Py is never zero on J, an existence theorem (discussed in Section 6.5) guarantees 
that solutions always exist over the interval /. Nevertheless, there is no general formula 
analogous to (6.2) for expressing these solutions in terms of Py), P,, P,, and R. Thus, even 
in this relatively simple generalization of (6.1), the theory is far from complete, except in 
special cases. If the coefficients are constants and if R is zero, all the solutions can be 
determined explicitly in terms of polynomials, exponential and trigonometric functions by 
the following theorem which was proved in Volume I (Theorem 8.7). 


THEOREM 6.2. Consider the differential equation 


(6.3) y’ +ay' + by=0, 
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where a and b are given real constants. Let d = a® — 4b. Then every solution of (6.3) on the 
interval (— 00, + 0) has the form 


(6.4) y =e 2 10,u,(X) + Cote(x)], 


where c, and c, are constants, and the functions u, and u, are determined according to the 
algebraic sign of d as follows: 

(a) Ifd = 0, then u,(x) = 1 and u(x) = x. 7 

(b) Ifd > 0, then u,(x) = e and u(x) = e-™, where k = 1/ 

(c) Ifd <0, then u,(x) = cos kx and u,(x) = sin kx, where k = L/—d. 


The number d = a? — 4b is the discriminant of the quadratic equation 
(6.5) r+tar+b=0. 


This is called the characteristic equation of the differential equation (6.3). Its roots are given 
By 
—a+vd —a—Vd 
SO 
Z 2 
The algebraic sign of d determines the nature of these roots. If d > 0 both roots are real 
and the solution in (6.4) can be expressed in the form 


y= ce +-c,e""*. 


If d <0, the roots r, and r, are conjugate complex numbers. Each of the complex 
exponential functions f,(x) = e”” and f,(x) = e”* is a complex solution of the differential 
equation (6.3). We obtain real solutions by examining the real and imaginary parts of /, 


and f,. Writing r, = —$a + ik, r, = —ta — ik, where k = 1./—d, we have 


fi(x) — eit — eal 2otke e 2/2 agg kx + je t*/? sin kx 
and 


fox) = ef = ete —tht = e882 eos kx — ie ***sin kx. 


The general solution appearing in Equation (6.4) is a linear combination of the real and 
imaginary parts of /,(x) and f,(x). 


6.3 Exercises 


These exercises have been selected from Chapter 8 in Volume I and are intended as a review of 
the introductory material on linear differential equations of first and second orders. 


Linear equations of first order. In Exercises 1, 2, 3, solve the initial-value problem on the specified 
interval. 


l. y’ — 3y = e** on (—~, +), with y = 0 when x =0. 
2. xy’ — 2y = x° on (0, +0), with y = 1 whenx = 1. 
3. y’ + ytanx = sin 2x on (—37, $7), with y = 2 when x =0. 
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4. If a strain of bacteria grows at a rate proportional to the amount present and if the population 
doubles in one hour, by how much will it increase at the end of two hours? 

5. A curve with Cartesian equation y = f(x) passes through the origin. Lines drawn parallel to 
the coordinate axes through an arbitary point of the curve form a rectangle with two sides on 
the axes. The curve divides every such rectangle into two regions A and B, one of which has an 
area equal to n times the other. Find the function f¢ 

6. (a) Let u be a nonzero solution of the second-order equation y” + P(x)y’ + Q@x)y =0. 
Show that the substitution y = wv converts the equation 


y” + P(x)y’ + O(x)y = R(x) 


into a first-order linear equation for v’. 
(b) Obtain a nonzero solution of the equation y” — 4y’ + x*(y’ — 4y) = 0 by inspection and 
use the method of part (a) to find a solution of 


y" —4y' + x2 (y' — 4y) = 2xe-7°/8 


such that y = 0 and y’ = 4 when x = 0. 


Linear equations of second order with constant coefficients. In each of Exercises 7 through 10, find 
all solutions on (— ©, +). 


7. yay Os 9. y’ —2y’ + 5y =0. 
8. y’ +4y =0. 10. y° +2y’+y =0. 


11. Find all values of the constant & such that the differential equation y” + ky = Ohas a nontrivial 
solution y = f;,(x) for which f,,(0) = f,(1) = 0. For each permissible value of k, determine the 
corresponding solution y = f,(x). Consider both positive and negative values of k. 

12. If (a, 5) is a given point in the plane and if m is a given real number, prove that the differential 
equation y” + k*y = 0 has exactly one solution whose graph passes through (a, b) and has 
slope m there. Discuss separately the case k = 0. 

13. In each case, find a linear differential equation of second order satisfied by u, and up. 

(a) u,(x) = e*, u(x) =e. 

(b) u,(x) = e**, u(x) = xe”, 

(c) u(x) = e~*/2 cos x, u(x) = e*? sin x. 

(d) u(x) = sin (2x + 1), u.(x) = sin (2x + 2). 
(e) u(x) = cosh x, u(x) = sinh x. 

14. A particle undergoes simple harmonic motion. Initially its displacement is 1, its velocity is 1 
and its acceleration is —12. Compute its displacement and acceleration when the velocity is V8. 


6.4 Linear differential equations of order n 


A linear differential equation of order 7 is one of the form 
(6.6) Po(xyy™ + Prayer to + Pay = RO). 


The functions Py), P,,..., P, multiplying the various derivatives of the unknown function 
y are called the coefficients of the equation. In our general discussion of the linear equation 
we shall assume that all the coefficients are continuous on some interval J. The word 
‘interval’ will refer either to a bounded or to an unbounded interval. 
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In the differential equation (6.6) the leading coefficient Py plays a special role, since it 
determines the order of the equation. Points at which Po(x) = 0 are called singular points 
of the equation. The presence of singular points sometimes introduces complications that 
require special investigation. To avoid these difficulties we assume that the function P, 
is never zero on J. Then we can divide both sides of Equation (6.6) by Py, and rewrite the 
differential equation in a form with leading coefficient 1. Therefore, in our general dis- 
cussion we shall assume that the differential equation has the form 


(6.7) yO +P (xy) +++ + PL(xX)y = R(x). 


The discussion of linear equations can be simplified by the use of operator notation. Let 
@ (J) denote the linear space consisting of all real-valued functions continuous on an interval 
J. Let @"(J) denote the subspace consisting of all functions f whose first derivatives 
tT sf",...,f'™ exist and are continuous on J. Let P,,...,P, be n given functions in 
@(J) and consider the operator L: @"(J) > @(J) given by 


L(f) = f(”) + Pyfe™ + eat ee + Poy 
The operator L itself is sometimes written as 
L= D?+P,D°'*+-°>+P,, 


where D* denotes the kth derivative operator. In operator notation the differential equation 
in (6.7) is written simply as 


(6.8) L(y) = R. 


A solution of this equation is any function y in @"(J) which satisfies (6.8) on the interval J. 
It is easy to verify that L(y, + y.) = L(y,) + Liz), and that L(cy) = cL(y) for every 

constant c. Therefore L is a /inear operator. This is why the equation L(y) = Ris referred 

to as a linear equation. The operator L is called a /inear differential operator of order n. 
With each linear equation L(y) = R we may associate the equation 


L(y) = 0, 


in which the right-hand side has been replaced by zero. This is called the homogeneous 
equation corresponding to L(y) = R. When Ris not identically zero, the equation L(y) = R 
is called a nonhomogeneous equation. We shall find that we can always solve the non- 
homogeneous equation whenever we can solve the corresponding homogeneous equation. 
Therefore, we begin our study with the homogeneous case. 

The set of solutions of the homogeneous equation is the null space N(L) of the operator 
L. This is also called the solution space of the equation. The solution space is a subspace 
of €@"(VJ). Although @”(J) is infinite-dimensional, it turns out that the solution space N(L) 
is always finite-dimensional. In fact, we shall prove that 


(6.9) dim M(L) =n, 
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where 7 is the order of the operator L. Equation (6.9) is called the dimensionality theorem 
for linear differential operators. The dimensionality theorem will be deduced as a conse- 
quence of an existence-uniqueness theorem which we discuss next. 


6.5 The existence-uniqueness theorem 


THEOREM 6.3. EXISTENCE-UNIQUENESS THEOREM FOR LINEAR EQUATIONS OF ORDER nn. Let 
P,,P,,...,P,, be continuous functions on an open interval J, and let L be the linear differ- 
ential operator 

Les D® Pp De set Pes 


If x9 € J and ifky, ky,...,k,_, aren given real numbers, then there exists one and only one 
function y = f(x) which satisfies the homogeneous differential equation L(y) = 0 on J and 
which also satisfies the initial conditions 


S (Xo) = kos f' (Xo) = 1, 0 sf OOP (%) = Kar. 


Note: The vector in n-space given by (f(x), f'(%o),--- > f''"-(%)) is called the 
initial-value vector of fat xy», Theorem 6.3 tells us that if we choose a point x9 in J and 
choose a vector in n-space, then the homogeneous equation L(y) = 0 has exactly one 
solution y = f(x) on J with this vector as initial-value vector at x). For example, when 
n = 2 there is exactly one solution with prescribed value f(x) and prescribed derivative 
f'(%o) at a prescribed point x9. 


The proof of the existence-uniqueness theorem will be obtained as a corollary of more 
general existence-uniqueness theorems discussed in Chapter 7. An alternate proof for the 
case of equations with constant coefficients is given in Section 7.9. 


6.6 The dimension of the solution space of a homogeneous linear equation 


THEOREM 6.4. DIMENSIONALITY THEOREM. Let L: 6"(J)—> G(J) be a linear differential 
operator of order n given by 


(6.10) L= D°+P,D™'+::+4+P,. 
Then the solution space of the equation L(y) = 0 has dimension n. 


Proof. Let V,, denote the n-dimensional linear space of n-tuples of scalars. Let T be the 
linear transformation that maps each function fin the solution space N(L) onto the initial- 
value vector of fat x), 


T(f) = (f(%0), F(X)» « «+ sf OP (X)) 


where xy is a fixed point inJ. The uniqueness theorem tells us that T(/) = 0 implies f = 0. 
Therefore, by Theorem 2.10, T is one-to-one on N(L). Hence T~ is also one-to-one and 
maps V,, onto N(L), and Theorem 2.11 shows that dim M(L) = dim V, =n. 
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Now that we know that the solution space has dimension n, any set of n independent 
solutions will serve as a basis. Therefore, as a corollary of the dimensionality theorem we 
have: 


THEOREM 6.5. Let L: @"(J) > ©(J) be a linear differential operator of ordern. Ifu,,..., 
u,, are n independent solutions of the homogeneous differential equation L(y) = 0 on J, then 
every solution y = f(x) on J can be expressed in the form 


(6.11) f(x) = ents). 


where c,,..., C, are constants. 


Note: Since all solutions of the differential equation L(y) = 0 are contained in formula 
(6.11), the linear combination on the right, with arbitrary constants c,,...,¢,, 1s 
sometimes called the general solution of the differential equation. 


The dimensionality theorem tells us that the solution space of a homogeneous linear 
differential equation of order 1 always has a basis of n solutions, but it does not tell us how 
to determine such a basis. In fact, no simple method is known for determining a basis of 
solutions for every linear equation. However, special methods have been devised for 
special equations. Among these are differential equations with constant coefficients to 
which we turn now. 


6.7 The algebra of constant-coefficient operators 


A constant-coefficient operator A is a linear operator of the form 
(6.12) A =a,D" + 4,D"? ++-*:+4,.,) +4,, 


where D is the derivative operator and a), a@,,...,4, are real constants. If a) ¥ 0 the 
operator is said to have order n. The operator A can be applied to any function y with n 
derivatives on some interval, the result being a function A(y) given by 


A(y) = ayy” + ayy) + cee —+ Any + Any. 


In this section, we restrict our attention to functions having derivatives of every order on 
(— oo, +0). The set of all such functions will be denoted by @® and will be referred to as 
the class of infinitely differentiable functions. If ye @® then A(y) is also in @®. 

The usual algebraic operations on linear transformations (addition, multiplication by 
scalars, and composition or multiplication) can be applied, in particular, to constant- 
coefficient operators. Let A and B be two constant-coefficient operators (not necessarily 
of the same order). Since the sum A + B and all scalar multiples AA are also constant- 
coefficient operators, the set of all constant-coefficient operators is a linear space. The 
product of A and B (in either order) is also a constant-coefficient operator. Therefore, 
sums, products, and scalar multiples of constant-coefficient operators satisfy the usual 
commutative, associative, and distributive laws satisfied by all linear transformations. 
Also, since we have D'D*’ = D*D" for all positive integers r and s, any two constant- 
coefficient operators commute; AB = BA. 
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With each constant-coefficient operator A we associate a polynomial p, called the 
characteristic polynomial of A. If A is given by (6.12), p4 is that polynomial which has the 
same coefficients as A. That is, for every real r we have 


Palr) = aor” + ar" +++ + a,. 


Conversely, given any real polynomial p, there is a corresponding operator A whose 
coefficients are the same as those of p. The next theorem shows that this association 
between operators and polynomials is a one-to-one correspondence. Moreover, this 
correspondence associates with sums, products, and scalar multiples of operators the 
respective sums, products, and scalar multiples of their characteristic polynomials. 


THEOREM 6.6. Let A and B denote constant-coefficient operators with characteristic 
polynomials p4 and pp, respectively, and let 4 be a real number. Then we have: 

(a) A = Bifand only if p4 = pep; 

(b) Pare = Pat Pp, 

(C) Pap = Pa’ PB: 

(d) pag =A" Pa- 


Proof. Weconsider part (a) first. Assume p, = pz. We wish to prove that A(y) = BY) 
for every yin @”. Since py = pz, both polynomials have the same degree and the same 
coefficients. Therefore A and B have the same order and the same coefficients, so A(}') = 
B(y) for every y in 6%. 

Next we prove that A = Bimpliesp, = pg. Therelation A = Bmeans that A(y) = By) 
for every yin@®. Take y = e™, wherer is aconstant. Since y“) = r*e™ for every k > 0, 
we have 


A(y)=pa(ne™ and By) =pp(re™. 


The equation A(y) = B(y) implies p4(r) = pp(r). Since r is arbitrary we must have 
Pa =Pp.~ This completes the proof of part (a). 
Parts (b), (c), and (d) follow at once from the definition of the characteristic polynomial. 


From Theorem 6.6 it follows that every algebraic relation involving sums, products, and 
scalar multiples of polynomials p, and pz also holds for the operators A and B. In 
particular, if the characteristic polynomial p, can be factored as a product of two or more 
polynomials, each factor must be the characteristic polynomial of some constant-coefficient 
operator, so, by Theorem 6.6, there is a corresponding factorization of the operator A. 
For example, if p4(r) = pa(n)pc(r), then A = BC. If p4(r) can be factored as a product of 
n linear factors, say 


(6.13) PA) = alr —n)ir— rss (r—7,), 
the corresponding factorization of A takes the form 


A =a,(D—1r,)(D—r,)+:*(D—1r,). 
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The fundamental theorem of algebra tells us that every polynomial p,(r) of degree 
n > 1 has a factorization of the form (6.13), wherer,,7ro,...,7,, are the roots of the equa- 
tion, 

par) = 0, 


called the characteristic equation of A. Each root is written as often as its multiplicity 
indicates. The roots may be real or complex. Since p,(r) has real coefficients, the complex 
roots occur in conjugate pairs, « + iB, «a —if, if B #0. The two linear factors 
corresponding to each such pair can be combined to give one quadratic factor r? — 2ar + 
a2 + B? whose coefficients are real. Therefore, every polynomial p,(r) can be factored as 
a product of linear and quadratic polynomials with real coefficients. This gives a corre- 
sponding factorization of the operator A as a product of first-order and second-order 
constant-coefficient operators with real coefficients. 


EXAMPLE |. Let A = D? —5D + 6. Since the characteristic polynomial p,(r) has the 
factorization r? — 5r + 6 = (r — 2)(r — 3), the operator A has the factorization 


D? —~ 5D + 6 = (D — 2)(D — 3). 


EXAMPLE 2. Let A = D* — 2D? + 2D* — 2D + 1. The characteristic polynomial p4(r) 
has the factorization 


r§6— 27 4+ 2? —2r +1 =(r—1)r— 124+ 1), 


so A has the factorization A = (D — 1)(D — 1)(D? + 1). 


6.8 Determination of a basis of solutions for linear equations with constant coefficients 
by factorization of operators 


The next theorem shows how factorization of constant-coefficient operators helps us to 
solve linear differential equations with constant coefficients. 


THEOREM 6.7. Let L be a constant-coefficient operator which can be factored as a product 
of constant-coefficient operators, say 


LT = A,A,°*:+ Ay. 


Then the solution space of the linear differential equation L(y) = 0 contains the solution space 
of each differential equation A;(y) = 0. In other words, 


(6.14) N(A;) & N(L) for eachi=1,2,...,k. 
Proof. If wis in the null space of the last factor A, we have A,(u) = 0 so 


L(u) = (AyAq** + Ay)(u) = (Ay + * + Aya) A,(u) = (Ar * * Ap_1)(0) = 0. 
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Therefore the null space of L contains the null space of the last factor A,. But since 
constant-coefficient operators commute, we can rearrange the factors so that any one of 
them is the last factor. This proves (6.14). 

If L(u) = 0, the operator L is said to annihilate u. Theorem 6.7 tells us that if a factor A; 
of Z annihilates u, then L also annihilates u. 

We illustrate how the theorem can be used to solve homogeneous differential equations 
with constant coefficients. We have chosen examples to illustrate different features, 
depending on the nature of the roots of the characteristic equation. 

CASE I. Real distinct roots. 

EXAMPLE |. Find a basis of solutions for the differential equation 
(6.15) (D? —7D + 6)y = 0. 

Solution. This has the form L(y) = 0 with 

L= D—7D+ 6 = (D — 1)(D — 2)(D + 3). 
The null space of D — 1 contains u,(x) = e*, that of D — 2 contains u,(x) = e*”, and that 
of D + 3 contains us(x) = e**. In Chapter 1 (p. 10) we proved that u,, u., us are inde- 


pendent. Since three independent solutions of a third order equation form a basis for the 
solution space, the general solution of (6.15) is given by 


y = cye* + coe + cge*. 


The method used to solve Example 1 enables us to find a basis for the solution space of 
any constant-coefficient operator that can be factored into distinct linear factors. 


THEOREM 6.8. Let L be a constant coefficient operator whose characteristic equation 
PLY) = 0 has n distinct real roots ry, r2,...,1,. Then the general solution of the differential 


>" ne 


equation L(y) = 0 on the interval (— 0, + ©) is given by the formula 


(6.16) y =) c,e". 
k=1 
Proof. We have the factorization 
L =a (D —1,)(D —1r.)+**(D—1,). 


Since the null space of (D — r,) contains u,(x) = e””, the null space of L contains the n 
functions 


(6.17) u,(x) =e", U(X) Se cag tt (x) He. 
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In Chapter 1 (p. 10) we proved that these functions are independent. Therefore they form 
a basis for the solution space of the equation L(y) = 0, so the general solution is given by 
(6.16). 


CASE IT. Real roots, some of which are repeated. 

If all the roots are real but not distinct, the functions in (6.17) are not independent and 
therefore do not form a basis for the solution space. If a root r occurs with multiplicity m, 
then (D — r)™ is a factor of L. The next theorem shows how to obtain m independent 
solutions in the null space of this factor. 


THEOREM 6.9. The m functions 
u,(x) = e”, U(X) SNE c(h) Sx 
are m independent elements annihilated by the operator (D — r)”™. 


Proof. The independence of these functions follows from the independence of the 
polynomials 1, x, x?,...,x™” "1. To prove that u,, vw.,...,u,, are annihilated by (D — r)™ 
we use induction on m. 

If m = | there is only one function, u,(x) = e™, which is clearly annihilated by (D — r). 
Suppose, then, that the theorem is true form — 1. This means that the functions u4,,..., 
U,,-1 are annihilated by (D — r)™"!. Since 


(D —r)™ = (D —r)(D —1r)™ 


the functions 4,,...,U,_; are also annihilated by (D — r)”. To complete the proof we 
must show that (D — r)” annihilates u,,. Therefore we consider 


(D — r)"u,, = (D — r)™1(D — r)(x™“1e"?). 
We have 


(D — r)\(x™ Ie) = D(x™ le) — rx™lere 
= (m — 1)x™ er 4+ x™ Ire — rxmlere 
= (m — 1)x™%e"* = (m — 1)u,,_1(x). 
When we apply (D — r)”"! to both members of this last equation we get 0 on the right 


since (D — r)”"' annihilates u,,,. Hence (D—r)"u,,=0 so u,, is annihilated by 
(D —r)™. This completes the proof. 


EXAMPLE 2. Find the general solution of the differential equation L(y) = 0, where 
L= D'— D?—8D+ 12. 


Solution. The operator L has the factorization 


L = (D — 2)(D + 3). 
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By Theorem 6.9, the two functions 
U(x) =e, g(x) = xe 


are in the null space of (D — 2)?. The function u,(x) = e~** is in the null space of (D + 3). 
Since u,, U,, Us are independent (see Exercise 17 of Section 6.9) they form a basis for the 
null space of L, so the general solution of the differential equation is 


y = cye* + coxe* + cge*. 


Theorem 6.9 tells us how to find a basis of solutions for any nth order linear equation 
with constant coefficients whose characteristic equation has only real roots, some of which 
are repeated. If the distinct roots are r,,r,,...,7, and if they occur with respective 
multiplicities m,, m,.,..., m,, that part of the basis corresponding to r, is given by the m, 
functions 

Ug, p(X) = xt Te, = where g=1,2,...,mp. 


As p takes the values 1,2,...,k we get m, +--+: + m, functions altogether. In Exercise 
17 of Section 6.9 we outline a proof showing that all these functions are independent. Since 
the sum of the multiplicities m, + +--+ + m, is equal to n, the order of the equation, the 
functions u, , form a basis for the solution space of the equation. 


EXAMPLE 3. Solve the equation (D® + 2D® — 2D* — D*)y = 0. 


Solution. We have D® + 2D® — 2D* — D? = D*(D — 1)(D + 1). The part of the 
basis corresponding to the factor D® is u,(x) = 1, uo(x) = x; the part corresponding to 
the factor (D — 1) is u3(x) = e*; and the part corresponding to the factor (D + 1)? is 
U,(x) = e*, u(x) = xe*, ug(x) = xe *. The six functions uw,,..., ug are independent 
so the general solution of the equation is 


Y= Cy H CQxX + Ce" + (Cy + C5xX + Cex*)e™. 


CASE III. Complex roots. 

If complex exponentials are used, there is no need to distinguish between real and complex 
roots of the characteristic equation of the differential equation L(y) = 0. If real-valued 
solutions are desired, we factor the operator L into linear and quadratic factors with real 
coefficients. Each pair of conjugate complex roots « + if, « — if corresponds to a 
quadratic factor, 


(6.18) D? — 24D + a? + B?. 
The null space of this second-order operator contains the two independent functions 
u(x) = e** cos Bx and v(x) = e** sin Bx. If the pair of roots « + if occurs with multi- 


plicity m, the quadratic factor occurs to the mth power. The null space of the operator 


[D? — 2aD + a? + BI” 
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contains 2m independent functions, 
u(x) = xt 1e%* cos Bx, v(x) = xe sin Bx, 7 i OS 


These facts can be easily proved by induction on m. (Proofs are outlined in Exercise 20 
of Section 6.9.) The following examples illustrate some of the possibilities. 


EXAMPLE 4, y" — 4y” + 13y’ =0. The characteristic equation, r? — 4r? + 13r =0, 
has the roots 0, 2 + 3i; the general solution is 


y = Cc; + e**(c, cos 3x + Cy sin 3x). 
EXAMPLE 5. y" — 2y” + 4y’ — 8y =0. The characteristic equation is 
rp — 24+ 4r—-8=(r—2)(r?2? +4 =0; 
its roots are 2, 2i, —2i, so the general solution of the differential equation is 
y =c,e* + c,cos 2x + cs sin 2x. 
EXAMPLE 6. y®) — Oy") + 34y" — 66y” + 65y’ — 25y =0. The characteristic equation 
can be written as 
(ry — 1)? — 4r + 5)? =0; 
its roots are 1,2 + i, 2 + i, so the general solution of the differential equation is 


y = cye* + e**[(cy + Cgx) COS xX + (Cy + C5X) Sin x]. 


6.9 Exercises 


Find the general solution of each of the differential equations in Exercises | through 12. 


1. y” —2y” —3y’ =0. 7. y' + loy =0. 

2. Ne yl = . 8. y' —y=0. 

3. y" + 4y" + 4y’ =0. 9. y@) + 4y™" + 8y” + By’ + 4y =0. 
4, y” —3y" +3y’ —y =0. 10. y) 4+ 2y” +y =0. 

5. y@) +. 4y" + Oy" +4y' +y =0. 11. yp 4 4y +4 4y” =0. 

6. y — 16y =0. 12. y) + 8y) + 16y” =0. 


13. If m is a positive constant, find that particular solution y = f(x) of the differential equation 


M 


y" — my” + my’ — my =0 


which satisfies the condition f(0) = f’(0) = 0, f"(0) = 1. 

14. A linear differential equation with constant coefficients has characteristic equation f(r) = 0. 
If all the roots of the characteristic equation are negative, prove that every solution of the differ- 
ential equation approaches zero as x — +00. What can you conclude about the behavior of all 
solutions on the interval [0, + ©) if all the roots of the characteristic equation are nonpositive? 


15. 


16. 


17. 
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In each case, find a linear differential equation with constant coefficients satisfied by all the 
given functions. 


(a) w(x) =e7, u(x) =e, g(x) = e?*, uy (x) =, 

(b) u,(x) = e2, u(x) = xe, Ug(x) = xe, 

(c) u(x) =1, U(x) = x, Us(x) = e”, u,(x) = xe. 

(d) u(x) =x, U(x) = e*, Us(x) = xe”. 

(€) u(x) =x?, u(x) = e*, g(x) = xe”. 

(f) u,(x) = e* cos 3x, U.(x) = e * sin 3x, u(x) =e, u,(x) = xe, 


(g) u(x) = cosh x, u.(x) = sinh x, u3(x) = x cosh x, u,(x) = xsinhx. 

(h) u,(x) = cosh x sin x, u.(x) = sinh x cos x, U3(x) =X. 

Letr,,..., 1, ben distinct real numbers, and let Q,,..., Q, be m polynomials, none of which 
is the zero polynomial. Prove that the n functions 


u,(x) = O,(x)e, ..., Un(x) = O,(xe""* 
are independent. 


Outline of proof. Use induction on n. For n = 1 and n = 2 the result is easily verified. 
Assume the statement is true for n = p and let c,,..., Cp 4, be p + 1 real scalars such that 


pt+l1 
> cxOxu(xe™ = 0. 
k=1 


Multiply both sides by e~"»+1” and differentiate the resulting equation. Then use the induction 

hypothesis to show that all the scalars c, are 0. An alternate proof can be given based on order 

of magnitude as x — +0, as was done in Example 7 of Section 1.7 (p. 10). 

Let m,, mz,..., m, be k positive integers, let r;, rg, ..., 7, be k distinct real numbers, and 

letn =m, +--:: + m,. For each pair of integers p, q satisfying 1 <p <k,1 <q < mp, let 
Ug, p(X) = xt lerr® , 

For example, when p = 1 the corresponding functions are 


Uy (x) =e”, Us (xX) = xe, 6. Um (x) = x™Te*, 


Prove that the n functions u,,, so defined are independent. [Hint: Use Exercise 16.] 


. Let L be a constant-coefficient linear differential operator of order n with characteristic poly- 


nomial p(r). Let L’ be the constant-coefficient operator whose characteristic polynomial is the 
derivative p’(r). For example, if L = 2D? —3D +1 then L’ =4D —3. More generally, 
define the mth derivative L‘”) to be the operator whose characteristic polynomial is the mth 
derivative p‘™(r). (The operator L‘” should not be confused with the mth power L”.) 

(a) If u has n derivatives, prove that 


“. plk(Q 
Gy = > Ps yf). 
k=0 ; 


(b) If u has n — m derivatives, prove that 


L‘™(y) = ye We for m=0,1,2,...,”, 
k=0 : 


where L!) = L. 
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19. Refer to the notation of Exercise 18. If u and v have n derivatives, prove that 


n 
L™® 
L(w) => yi) , 
k=0 ° 


[Hint: Use Exercise 18 along with Leibniz’s formula for the kth derivative of a product: 


(uv)™ = s (Huy lr) | 


r=0 


20. (a) Let p(t) = q(t)™r(t), where q and r are polynomials and m is a positive integer. Prove that 
p(t) =q(t)™ 1s(t), where s is a polynomial. 
(b) Let L be a constant-coefficient operator which annihilates u, where u is a given function of 
x. Let M = L™, the mth power of L, where m > 1. Prove that each of the derivatives M ce 


M”",..., M‘™-) also annihilates u. 

(c) Use part (b) and Exercise 19 to prove that M annihilates each of the functions u, xu,..., 
xm™ly, 

(d) Use part (c) to show that the operator (D? — 2«D + a + 6?)™ annihilates each of the 
functions x%e** sin Bx and x%e** cos Bx forg =1,2,...,m—1. 


21. Let L be a constant-coefficient operator of order n with characteristic polynomial p(r). If « 
is constant and if u has n derivatives, prove that 


re pk) 
L(e**u(x)) = eye u'™ (x). 
k=0 


6.10 The relation between the homogeneous and nonhomogeneous equations 


We return now to the general linear differential equation of order n with coefficients that 
are not necessarily constant. The next theorem describes the relation between solutions of 
a homogeneous equation L(y) = 0 and those of a nonhomogeneous equation L(y) = R(x). 


THEOREM 6.10. LetL:@,,(J)— @(J) be a linear differential operator of ordern. Letu,,..., 
u,, be n independent solutions of the homogeneous equation L(y) = 0, and let y, be a particular 
solution of the nonhomogeneous equation L(y) = R, where RE G(J). Then every solution 
y =f (x) of the nonhomogeneous equation has the form 


(6.19) f(x) = yy(x) + Sean), 


wheré C1,... 4 C, are constants. 


Proof. By linearity we have L(f— y,) = L(f) — LO) = R-— R=0. Therefore 
f — yx is in the solution space of the homogeneous equation L(y) = 0, so f — jy; is a linear 
combination of u,,...,u,, say f— yy = cu, +°++ + ¢,u,. This proves (6.19). 


Since all solutions of L(y) = R are found in (6.19), the sum on the right of (6.19) (with 
arbitrary constants c,,C,,...,C,) is called the general solution of the nonhomogeneous 
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equation. Theorem 6.10 states that the general solution of the nonhomogeneous equation 
is obtained by adding to y, the general solution of the homogeneous equation. 


Note: Theorem 6.10 has a simple geometric analogy which helps give an insight into 
its meaning. To determine all points on a plane we find a particular point on the plane 
and add to it all points on the parallel plane through the origin. To find all solutions of 
L(y) = R we find a particular solution and add to it all solutions of the homogeneous 
equation L(y) = 0. The set of solutions of the nonhomogeneous equation is analogous 
to a plane through a particular point. The solution space of the homogeneous equation 
is analogous to a parallel plane through the origin. 


To use Theorem 6.10 in practice we must solve two problems: (1) Find the general 
solution of the homogeneous equation L(y) = 0, and (2) find a particular solution of the 
nonhomogeneous equation L(y) = R. In the next section we show that we can always 
solve problem (2) if we can solve problem (1). 


6.11 Determination of a particular solution of the nonhomogeneous equation. The method 
of variation of parameters 


We turn now to the problem of determining one particular solution y, of the nonhomo- 
geneous equation L(y) = R. We shall describe a method known as variation of parameters 
which tells us how to determine y, if we know n independent solutions u,,...,u, of the 
homogeneous equation L(y) = 0. The method provides a particular solution of the form 


(6.20) V1 = Vu, + as + UAUn> 


where v,,..., UV, are functions that can be calculated in terms of u,,..., u, and the right- 
hand member R. The method leads to a system of n linear algebraic equations satisfied by 
the derivatives v|,...,v;,. This system can always be solved because it has a nonsingular 
coefficient matrix. Integration of the derivatives then gives the required functions v,,..., 
v,- Lhe method was first used by Johann Bernoulli to solve linear equations of first order, 
and then by Lagrange in 1774 to solve linear equations of second order. 

For the th order case the details can be simplified by using vector and matrix notation. 
The right-hand member of (6.20) can be written as an inner product, 


(6.21) y= (v4), 
where v and u are n-dimensional vector functions given by 
OO) 2245505) % = (hs. a n0eg ls 


We try to choose v so that the inner product defining y, will satisfy the nonhomogeneous 
equation L(y) = R, given that L(u) = 0, where L(u) = (L(m),..., L(u,)). 
We begin by calculating the first derivative of y,. We find 


(6.22) Yi = (v, u) + (0, u). 


158 Linear differential equations 
We have n functions v,,..., v, to determine, so we should be able to put 1 conditions on 
them. If we impose the condition that the second term on the right of (6.22) should vanish, 
the formula for y, simplifies to 
y,=(v, uv’), provided that (v’, vu) = 0. 
Differentiating the relation for y, we find 
yl = (0, u") + 0, uw’). 
If we can choose v so that (v’, u’) = 0 then the formula for y{ also simplifies and we get 
yi = (0, u"), provided that also (v’, u’) = 0. 
If we continue in this manner for the first n — 1 derivatives of y, we find 


yr) = (v, uu"), provided that also (v’, u'"-?’) = 0. 


So far we have put n — | conditions on v. Differentiating once more we get 


ye = (v, a”) ui (v’, 1). 


This time we impose the condition (v’, u“—))) = R(x), and the last equation becomes 


yy = (v, u'™) + R(x), provided that also (v’, u'"~Y) = R(x). 


Suppose, for the moment, that we can satisfy the m conditions imposed on v. Let L = 
D" + P,(x)D""1! +-+-+++P,(x). When we apply L to ), we find 


L(y) = ig + P,(x)yi" +e + P,(x)y1 
= {(v, uw) + R(x)} + Prx)(v, ul") + + PaCx)(v, 4) 
= (v, L(u)) + R(x) = (v, 0) + R(x) = R(X). 
Thus L(y,) = R(x), so y, is a solution of the nonhomogeous equation. 
The method will succeed if we can satisfy the n conditions we have imposed on v. These 


conditions state that (v', u“™)) = 0 fork =0,1,...,n — 2, and that (v', u-))) = R(x). 
We can write these n equations as a single matrix equation, 


0 
(6.23) W(x)v'(x) = R(x) : ; 
l 
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where v'(x) is regarded as an n Xx 1 column matrix, and where W is the n x n matrix 
function whose rows consist of the components of u and its successive derivatives: 


u, Us eee u, 
uy Us u 
W= 
we!) co ao) 


The matrix W is called the Wronskian matrix of u,,...,u,, after J. M. H. Wronski 
(1778-1853). 

In the next section we shall prove that the Wronskian matrix is nonsingular. Therefore 
we can multiply both sides of (6.23) by W(x)! to obtain 


0 
v(x) = R(x)W(x)" ; , 
l 


Choose two points c and x in the interval J under consideration and integrate this vector 
equation over the interval from c to x to obtain 


0 
v(x) = v(¢) + | ROW] 3 | dt = wfc) + 2(x), 
1 
where 
0 
z(x) = | R() W(t) : dt. 


l 
The formula y, = (u, v) for the particular solution now becomes 
yi = (uy, v) = (yu, v(c) + z) = (u, v(c)) + (Y, Z). 


The first term (uw, v(c)) satisfies the homogeneous equation since it is a linear combination of 
u,,...,U,. Therefore we can omit this term and use the second term (u, z) as a particular 
solution of the nonhomogeneous equation. In other words, a particular solution of 
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L(y) = R is given by the inner product 
0 


(u(x), z(x)) = (uc. | "R(D)W(t)2 at). 
1 


Note that it is not necessary that the function R be continuous on the interval J. All that is 
required is that R be integrable on [c, x]. 
We can summarize the results of this section by the following theorem. 


THEOREM 6.11. Letu,,...,u, ben independent solutions of the homogeneous nth order 
linear differential equation L(y) = 0 on an interval J. Then a particular solution y, of the 
nonhomogeneous equation L(y) = R is given by the formula 


yi(x) = Yy(o)o4(x) ) 


where 0,,...,0U, are the entries in then x 1 column matrix v determined by the equation 
0 
Cen) v(x) = | R(t)W ty’ : dt. 
C 
l 
In this formula, W is the Wronskian matrix of u,,...,U,, and c is any point in J. 


Note: The definite integral in (6.24) can be replaced by any indefinite integral 
0 


| R(x)W(xy ; dx . 
l 


EXAMPLE |. Find the general solution of the differential equation 


on the interval (— 00, +00). 


Solution. The homogeneous equation, (D? — 1)y = 0 has the two independent solutions 
U,(x) = e*, uo(x) = e-*. The Wronskian matrix of uv, and u, is 


W(x) = 
e* = e * 
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Since det W(x) = —2, the matrix is nonsingular and its inverse is given by 


W(x)! =--< ; 


ae 0 || =e" 
mle 


R(x)W a 1 2 a 1+e 
Oe) 1 ~ 21+¢ et 7 _ et 


Integrating each component of the vector on the right we find 


Therefore 


and we have 


ne) = | i dx = [(e*—1+ ) dx = —e* — x + log (l + €*) 
1+ e° 1+ e’ 
and 


V(x) = { eae dx = —log(1 + e”). 


Therefore the general solution of the differential equation is 


Y = CyUy(X) + Colle(X) + Vy(x)uy(X) + 0_(x)ug(x) 
= c,e* + coe~* — 1 — xe* + (e* — e*) log (1 + e”). 


6.12 Nonsingularity of the Wronskian matrix of n independent solutions of a homogeneous 
linear equation 


In this section we prove that the Wronskian matrix W of n independent solutions 
u,,...,U, Of a homogeneous equation L(y) = 0 is nonsingular. We do this by proving 
that the determinant of W is an exponential function which is never zero on the interval J 
under consideration. 

Let w(x) = det W(x) for each x in J, and assume that the differential equation satisfied 
by u,,...,u, has the form 


(6.25) YO $ Py(x)yD $+ + PCy = 0. 
Then we have: 
THEOREM 6.12. The Wronskian determinant satisfies the first-order differential equation 


(6.26) w + P,(x)w = 0 
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on J. Therefore, if cE J we have 

(6.27) w(x) = w(c) exp ie P,(t) at| (Abel’s formula). 
Moreover, w(x) # 0 for all x in J. 


Proof. Let u be the row-vector u = (u,,...,u,). Since each component of u satisfies 
the differential equation (6.25) the same is true of u. The rows of the Wronskian matrix W 
are the vectors u, u’,..., u”-), Hence we can write 


w = det W = det (u,u’,...,u'”?). 


The derivative of w is the determinant of the matrix obtained by differentiating the last row 
of W (see Exercise 8 of Section 3.17). That is 


w -= det (u,u’,..., ul"), ul), 
Multiplying the last row of w by P,(x) we also have 
Pi(x)w = det (u,u’,... , ul"), Py(x)ul"-). 
Adding these last two equations we find 
w + P,(x)w = det (u,u’,..., ul), u™ + Py(x)u). 


But the rows of this last determinant are dependent since u satisfies the differential equation 
(6.25). Therefore the determinant is zero, which means that w satisfies (6.26). Solving 
(6.26) we obtain Abel’s formula (6.27). 

Next we prove that w(c) ¥ 0 for some cin J. We do this by a contradiction argument. 
Suppose that w(t) = 0 for all tin J. Choose a fixed ¢ in J, say t = fy, and consider the 
linear system of algebraic equations 

W(t))X = O, 


where X is a column vector. Since det W(t)) = 0, the matrix W(t)) is singular so this 
system has a nonzero solution, say ¥ = (c,,...,C,) # (0,...,0). Using the components 
of this nonzero vector, let f be the linear combination 


S(t) = q(t) + °° + ¢,u,(t). 


The function f so defined satisfies L(f) = 0 on J since it is a linear combination of 
u,,...,u,. The matrix equation W(t))X = O implies that 


F (to) =f (to) = 0 HFM (Eo) = 0. 
Therefore f has the initial-value vector O at t = fy so, by the uniqueness theorem, /f is the 
zero solution. This means c; = +--+ =c, =0, which is a contradiction. Therefore 


w(t) ¥ 0 for some tin J. Taking c to be this ¢ in Abel’s formula we see that w(x) ¥ 0 for 
all x in J. This completes the proof of Theorem 6.12. 
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6.13 Special methods for determining a particular solution of the nonhomogeneous equation. 
Reduction to a system of first-order linear equations 


Although variation of parameters provides a general method for determining a particular 
solution of L(y) = R, special methods are available that are often easier to apply when the 
equation has certain special forms. For example, if the equation has constant coefficients 
we can reduce the problem to that of solving a succession of linear equations of first order. 
The general method is best illustrated with a simple example. 


EXAMPLE |. Find a particular solution of the equation 
(6.28) (D — 1)(D — 2)y = xe, 
Solution. Let u = (D— 2)y. Then the equation becomes 
(D — 1)u = xe™**, 


This is a first-order linear equation in u which can be solved using Theorem 6.1. A particular 


solution is 
1 a2 
u= der os 


Substituting this in the equation u = (D — 2)y we obtain 
(D —2)y = he, 


a first-order linear equation for y. Solving this by Theorem 6.1 we find that a particular 
solution (with y,(0) = 0) is given by 


y1(x) = 4e* li et dt. 


Although the integral cannot be evaluated in terms of elementary functions we consider 
the equation as having been solved, since the solution is expressed in terms of integrals of 
familiar functions. The general solution of (6.28) is 


eo 48 
y = ce” + c,e** + de” I, edt. 


6.14 The annihilator method for determining a particular solution of the nonhomogeneous 
equation 


We describe next a method which can be used if the equation L(y) = R has constant 
coefficients and if the right-hand member R is itself annihilated by a constant-coefficient 
operator, say A(R) = 0. In principle, the method is very simple. We apply the operator 
A to both members of the differential equation L(y) = R and obtain a new equation 
AL(y) = 0 which must be satisfied by all solutions of the original equation. Since AL is 
another constant-coefficient operator we can determine its null space by calculating the 
roots of the characteristic equation of AL. Then the problem remains of choosing from 
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this null space a particular function y, that satisfies L(y,) = R. The following examples 
illustrate the process. 


EXAMPLE |. Find a particular solution of the equation 
(D* — l6)y=xt*+x41. 


Solution. The right-hand member, a polynomial of degree 4, is annihilated by the 
operator D°. Therefore any solution of the given equation is also a solution of the equation 


(6.29) D°(D* — 16)y = 0. 


The roots of the characteristic equation are 0, 0, 0,0, 0, 2, —2, 27, —2i, so all the solutions 
of (6.29) are to be found in the linear combination 


V = Cy + Cox + Cyx® + C4x? + 5x4 + Coe? + c7e7* + Cg COS 2x + Cg Sin 2x. 
We want to choose the c; so that L(y) = x* + x +1, where L = D‘ — 16. Since the last 
four terms are annihilated by L, we can take cy = cz = Cg = Cg = O and try to find 
C,,..., Cs SO that 


Ley + Cox + 3x? + cyx? + ¢5x*) = xt +x4+1. 


In other words, we seek a particular solution y, which is a polynomial of degree 4 satisfying 
L(y,) = xt +x +1. To simplify the algebra we write 


l6y, = ax4 + bx? + cx? + dx +e. 


This gives us l6y\*) = 24a, so y\!) = 3a/2. Substituting in the differential equation 
L(),) = x* + x + 1, we must determine a, b, c, d, e to satisfy 


3a — axt — bx® — cx? — dx —e=xi4+x4+1. 
Equating coefficients of like powers of x we obtain 
a=-—-l, b=c=0, d=-l, e=-}3 


so the particular solution y, is given by 


EXAMPLE 2. Solve the differential equation y” — Sy’ + 6y = xe”. 


Solution. The differential equation has the form 


(6.30) Liy)=R, 
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where R(x) = xe” and L = D? —5D+6. The corresponding homogeneous equation 
can be written as 


(D — 2)(D — 3)jy=0; 


it has the independent solutions u,(x) = e””, u.(x) = e”. Now we seek a particular 
solution y, of the nonhomogeneous equation. We recognize the function R(x) = xe* 
as a solution of the homogeneous equation 


(D — 1)*y =0. 


Therefore, if we operate on both sides of (6.30) with the operator (D — 1)? we find that any 
function which satisfies (6.30) must also satisfy the equation 


(D — 1)(D — 2)(D — 3)y = 0. 


This differential equation has the characteristic roots 1, 1, 2, 3, so all its solutions are to be 
found in the linear combination 


y = ae* + bxe* + ce + de®”, 


where a, b, c, d are constants. We want to choose a, b, c, d so that the resulting solution 
y, satisfies L(y,) = xe*. Since L(ce** + de®”) = 0 for every choice of c and d, we need only 
choose a and b so that L(ae” + bxe*) = xe” and take c= d=0. If we put 


y, = ae” + bxe’, 
we have 
D(\1) = (a + Dje* + bxe*, D?(y,) = (a + 2b)e* + bxe*, 


so the equation (D? — 5D + 6)y, = xe” becomes 
(Qa — 3b)e” + 2bxe* = xe”. 


Canceling e” and equating coefficients of like powers of x we find a = 23,5 = 3. Therefore 
y, = 2e” + $xe* and the general solution of L(y) = R is given by the formula 


y = ce" + coe** + ge” + 3xe". 


The method used in the foregoing examples is called the annihilator method. It will 
always work if we can find a constant coefficient operator A that annihilates R. From our 
knowledge of homogeneous linear differential equations with constant coefficients, we know 
that the only real-valued functions annihilated by constant-coefficient operators are linear 
combinations of terms of the form 

xe, xe Cos Bx, ox e** Sin Bx, 
where m is a positive integer and « and f are real constants. The function y = x™~le* 
is a solution of a differential equation with a characteristic root « having multiplicity m. 
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Therefore, this function has the annihilator (D — «)”. Each of the functions y = 
x™1e** cos x and y = x™~1e* sin 6x is a solution of a differential equation with complex 
characteristic roots « + if, each occurring with multiplicity m, so they are annihilated by 
the operator [D? — 2aD + (a? + 6*)]”. For ease of reference, we list these annihilators 
in Table 6.1, along with some of their special cases. 


TABLE 6.1 

Function Annihilator 
y — xml D™ 
y=e" D—-« 
y= x™-1ler (D pee a)” 
y=cosfpx or y=sin px D* + 
y=x™1 cos px or y=x™" sin Bx (D? + B?)™ 
y=e*cosBx or y=e**sin Bx D? — 24D + (a? + 6?) 
y = x™ le cos Bx or y=x™ Te sin Bx [D? — 24D + (a? + B?)]™ 


Although the annihilator method is very efficient when applicable, it is limited to 
equations whose right members R have a constant-coefficient annihilator. If R(x) has the 
form e*, log x, or tanx, the method will not work; we must then use variation of 
parameters or some other method to find a particular solution. 


6.15 Exercises 


In each of Exercises 1 through 10, find the general solution on the interval (— ©, +). 


1. y -y =x, 6. y” hed. 

2. yl" — 4y Ea gee. 7. yn" —y' = et 4 et, 

3. y" + 2y’ = 3xe*. 8. y" + 3y" + 3y’ +y =xe™. 
4. y" +4y =sinx. 9. yy +y = xe* sin 2x. 

5.y —2y +y =e +e. 10. y) — y = xe, 


11. If a constant-coefficient operator A annihilates f and if a constant-coefficient operator B 
annihilates ¢, show that the product AB annihilates f + g. 

12. Let A be a constant-coefficient operator with characteristic polynomial p,. 
(a) Use the annihilator method to prove that the differential equation A(y) = e*” has a 


particular solution of the form 
ere 


a> Palo) 


if « is not a zero of the polynomial p,. 


(b) If « is a simple zero of p, (multiplicity 1), prove that the equation A(y) = e*%* has the 
particular solution 
xe%% 


Pig() 


vi 


(c) Generalize the results of (a) and (b) when « is a zero of p, with multiplicity m. 


13. 


14. 


15. 
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Given two constant-coefficient operators A and B whose characteristic polynomials have no 
zeros in common. Let C = AB. 

(a) Prove that every solution of the differential equation C(y) = 0 has the form y = y, + yo, 
where A(y,) = 0 and By.) = 0. 

(b) Prove that the functions y, and yp, in part (a) are uniquely determined. That is, for a 
given y satisfying C(y) = 0 there is only one pair y, , y2 with the properties in part (a). 

If L(y) = y” + ay’ + by, where a and 5b are constants, let f be that particular solution of 
L(y) = 0 satisfying the conditions f(0) = 0 and f’(0) = 1. Show that a particular solution 
of L(y) = R is given by the formula 


yi(x) = [ * F(x — t)R(t) dt 


for any choice of c. In particular, if the roots of the characteristic equation are equal, say r; = 
r. = m, show that the formula for y,(x) becomes 


yx) =e™ [°c — the~™* R(t) dt. 


Let Q be the operator “‘multiplication by x.’ That is, Q(y)(x) = x - y(x) for each yin class 6 
and each real x. Let I denote the identity operator, defined by /(y) = y for each yin @®. 

(a) Prove that DO —QD =I. 

(b) Show that D?Q — QD? is a constant-coefficient operator of first order, and determine this 
operator explicitly as a linear polynomial in D. 

(c) Show that D?Q — QD? is a constant-coefficient operator of second order, and determine 
this operator explicitly as a quadratic polynomial in D. 

(d) Guess the generalization suggested for the operator D"Q — QD", and prove your result 
by induction. 


In each of Exercises 16 through 20, find the general solution of the differential equation in the 
given interval. 


16. y 


" 


—y=I1/x, (0, +0). 


he Tr 
17. y + 4y =sec 2x, ( ) 


-44 


" wT 
18. y" — y =sec? x — secx, ( ), 


- 373 


19. y” —2y +y =e%(e7 —1)?, (—w, +0). 


2x 


20. y” — Ty" + l4y’ — 8y = log x, (0, +0). 


6.16 Miscellaneous exercises on linear differential equations 


l. 


An integral curve y = u(x) of the differential equation y" — 3y’ — 4y = Ointersects an integral 
curve y = v(x) of the differential equation y”+ 4y’ — Sy = 0 at the origin. Determine the 
functions u and v if the two curves have equal slopes at the origin and if 


[v(x]? 5 


im a 
ao U(x) 6 


An integral curve y = u(x) of the differential equation y” — 4y’ + 29y = Ointersects an integral 
curve y = v(x) of the differential equation y” + 4y’ + 13y = 0 at the origin. The two curves 
have equal slopes at the origin. Determine u and v if u’(7/2) = 1. 
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. Given that the differential equation y” + 4xy’ + Q(x)y = 0 has two solutions of the form 


yy. = u(x) and y,. = xu(x), where u(0) = 1. Determine both u(x) and Q(x) explicitly in terms 
of x. 


. Let L(y) = y” + Pyy’ + Pay. To solve the nonhomogeneous equation L(y) = R by variation 


of parameters, we need to know two linearly independent solutions of the homogeneous 
equation. This exercise shows that if one solution u, of L(y) = 0 is known, and if uw, is never 
zero on an interval J, a second solution u, of the homogeneous equation is given by the formula 


t 
we) = 40 | Tapp 


where Q(x) = eSPil@)d@, and c is any point in J. These two solutions are independent on J. 
(a) Prove that the function u, does, indeed, satisfy L(y) = 0. 
(b) Prove that uw, and u, are independent on J. 


. Find the general solution of the equation 


xy" —2(x + ly’ + (x + 2)y = xe 


for x > 0, given that the homogeneous equation has a solution of the form y = e™*. 


. Obtain one nonzero solution by inspection and then find the general solution of the differential 


equation 
(y" — 4y’') + x20)’ — 4y) =0. 


. Find the general solution of the differential equation 


4x*y" + 4xy’ —y =0, 


given that there is a particular solution of the form y = x™ for x > 0. 


. Find a solution of the homogeneous equation by trial, and then find the general solution of the 


equation 
x(1 — x)y” — (1 — 2x)y’ + (xX? — 3x + Dy = (1 — x). 


. Find the general solution of the equation 


(2x — 3x%)y" + 4y’ + Oxy =0, 


given that it has a solution that is a polynomial in x. 


. Find the general solution of the equation 


x*(1 — x)y” + 2x(2 — xy’ +21 + xy = x, 


given that the homogeneous equation has a solution of the form y = x°. 
Let g(x) = Jve'/t dt if x >0. (Do not attempt to evaluate this integral.) Find all values of 
the constant a such that the function f defined by 


f@ = 1 e%9 (x) 
x 
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satisfies the linear differential equation 
x*y" + (3x — x*)y’ + (1 — x — e*)y =0. 
Use this information to determine the general solution of the equation on the interval (0, + 00), 


6.17 Linear equations of second order with analytic coefficients 


A function fis said to be analytic on an interval (x) — r, X) + r) if f has a power-series 
expansion in this interval, 


f(x) = Yay(x — x)", 


convergent for |x — x9| <r. If the coefficients of a homogeneous linear differential 
equation 
Yt Py(xyOD $+ + PyCay = 0 


are analytic in an interval (x9 — r, X¥» +r), then it can be shown that there exist n inde- 
pendent solutions u,,...,u,, each of which is analytic on the same interval. We shall 
prove this theorem for equations of second order and then discuss an important example 
that occurs in many applications. 


THEOREM 6.13. Let P, and P, be analytic on an open interval (x) — r, X) +r), say 


P,(x) = > d,s — Xo)”, P,(x) = YenG — Xo)”. 
Then the differential equation 
(6.31) y+ Pix)’ + Po(x)y = 0 
has two independent solutions u, and u, which are analytic on the same interval. 


Proof. We try to find a power-series solution of the form 


(6.32) y =Sa,(x — x)", 


convergent in the given interval. To do this, we substitute the given series for P, and P, in 
the differential equation and then determine relations which the coefficients a, must satisfy 
so that the function y given by (6.32) will satisfy the equation. 

The derivatives y’ and y” can be obtained by differentiating the power series for y term 
by term (see Theorem 11.9 in Volume I). This gives us 


na,(x — Xo)" = 2 (n Fi L)anyilx =X) 


y! = Lm(n — a,x = x9" = Yn $l + Dayaalx — x)". 


n=2 


170 Linear differential equations 


The products P,(x)y’ and P,(x)y are given by the power series 


Pay” = E (E(k + Vnsabyaa) (e — 0)" 
and ne 


n=0 


P(x)y = 5 ( Y axe] (x — Xo)”. 


When these series are substituted in the differential equation (6.31) we find 


2 (n + 2)(n + Idnpe + 2 [tk + 1)ayp1Br—n + AeCn—n] (X — Xo)” = 9. 

n= ee 

Therefore the differential equation will be satisfied 1f we choose the coefficients a, so that 
they satisfy the recursion formula 


(6.33) (n+ 20+ Naqi2 = — SUK Day bnn + Ona 


for n=0,1,2,.... This formula expresses a,,,. in terms of the earlier coefficients 
Ay, @,,...,4,,;, and the coefficients of the given functions P, and P,. We choose arbitrary 
values of the first two coefficients a, and a, and use the recursion formula to define the 
remaining coefficients a,,a3,..., in terms of a) and a,. This guarantees that the power 
series in (6.32) will satisfy the differential equation (6.31). The next step in the proof is to 
show that the series so defined actually converges for every x in the interval (x) — r, X) +7). 
This is done by dominating the series in (6.32) by another power series known to converge. 
Finally, we show that we can choose a, and a, to obtain two independent solutions. 

We prove now that the series (6.32) whose coefficients are defined by (6.33) converges in 
the required interval. 

Choose a fixed point x, ¥ x, in the interval (x) — r, x) + r) and let ¢t = |x, — x). 
Since the series for P, and P, converge absolutely for x = x, the terms of these series are 
bounded, say 

[b,| t* << My and lel t* < Me, 


for some M, > 0, M, >0. Let M be the larger of M@, and tM,. Then we have 


M M 
lbxl < ye and len] < jee 


The recursion formula implies the inequality 


< M M 
(n+ 2)(n + 1) Laysal <> {K+ 1) lagaal og + lal Soa} 
k=0 


M n nr 
= zApXC $ WD ldgaal t+ > ages! + lagl — lanaal rl 
l k=0 k=0 
M n M n+1 
< MIS i 2) lays! alae as = yet dé + 1) |a,| t* 
k=0 k=0 


t Those readers not familiar with multiplication of power series may consult Exercise 7 of Section 6.21. 
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Now let Ay = |a|, A; = |a,|, and define A,, .A,,... successively by the recursion 
formula 
M n+1 
(6.34) (n+ 20 + DAnys = Dk + DA" 
t k=0 


forn > 0. Then |a,| < A, for all n > 0, so the series > a,,(x — xo)” is dominated by the 
series > A,, |x — Xo|”. Now we use the ratio test to show that > A, |x — x9|" converges if 
|x — Xo| < ¢. 

Replacing n by n — 1 in (6.34) and subtracting ¢—' times the resulting equation from (6.34) 
we find that (n + 2)(n + 1)A,4. — t'(n + 1)nA,,, = M(n + 2)A,,,. Therefore 


(n+ 1)n + (n+ 2)Mt 
(n + 2)(n + 1)t 


Anse = nil ) 


and we find 
Ansys |x — X9l"** —_ (n+ 1n+ (n+ 2)Mt 


Se ae ee 


Any |X — x (n + 2)(n + I)t 


as n> oo. This limit is less than 1 if |x — x9| <¢. Hence > a,(x — x9)" converges if 
|x — X9| << ¢. But since t = |x, — x,| and since x, was an arbitrary point in the interval 
(x9 — 1, Xo +r), the series > a,(x — xo)" converges for all x in (x) —r, x9 +17). 

The first two coefficients aj and a, represent the initial values of y and its derivative at the 
point xy. If we let u, be the power-series solution with aj = 1 and a, = 0, so that 


u,(Xo) = 1 and ui(Xo) = 0, 
and let uv, be the solution with aj = 0 and a, = 1, so that 
U2(Xo) = 0 and U(X) = 1, 


then the solutions u, and u, will be independent. This completes the proof. 


6.18 The Legendre equation 


In this section we find power-series solutions for the Legendre equation, 
(6.35) (1 — x*)y" — 2xy’ + a(a+ Dy =0, 


where « is any real constant. This equation occurs in problems of attraction and in heat- 
flow problems with spherical symmetry. When « is a positive integer we shall find that the 
equation has polynomial solutions called Legendre polynomials. These are the same 
polynomials we encountered earlier in connection with the Gram-Schmidt process (Chapter 
1, page 26). 

The Legendre equation can be written as 


[x? — Dy) = a(a + Dy, 
which has the form 


T(y) = Ay, 


172 Linear differential equations 


where T is a Sturm-Liouville operator, T(f) = (pf’)’, with p(x) = x? —1 and A= 
a(a + 1). Therefore the nonzero solutions of the Legendre equation are eigenfunctions of 
T belonging to the eigenvalue «(a + 1). Since p(x) satisfies the boundary conditions 


pil) = p(-1) = 0, 


the operator T is symmetric with respect to the inner product 


(f, 8) = ie: (x)g(x) dx. 


The general theory of symmetric operators tells us that eigenfunctions belonging to distinct 
eigenvalues are orthogonal (Theorem 5.3). 

In the differential equation treated in Theorem 6.13 the coefficient of y” is 1. The 
Legendre equation can be put in this form if we divide through by 1 — x?. From (6.35) we 
obtain 

y" + Py(x)y’ + Proxy =0, 
where 
P,(x) = -—, and Pix) = 4D, 
if x2 #1. Since 1/(1 — x?) = >? x” for |x| <1, both P, and P, have power-series 
expansions in the open interval (—1, 1) so Theorem 6.13 is applicable. To find the recursion 
formula for the coefficients it is simpler to leave the equation in the form (6.35) and try to 
find a power-series solution of the form 


Therefore we have 
fo @) [0 @) 
2xy" => 2na,x” = > 2na,x". 
n=1 n=0 


and 


(1 — x*)y” = > n(n — 1a,x”? — ¥ n(n — 1a,x” 
n=2 


=2 


= S (n + 2)(n + L)anyox” = y n(n a I)a,x" 


= aC + 2)(n + Dag.» — n(n — 1a, ]x". 


If we substitute these series in the differential equation (6.35), we see that the equation 
will be satisfied if, and only if, the coefficients satisfy the relation 


(n + 2)(n + 1)a,,. —n(n — la, — 2na, + a(a + la, = 0 
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for alln > 0. This equation is the same as 


(n + 2)(n + Nags — (n — a(n +: 1 + aa, = 0, 

or 

(6.36) a (a — nla +n +1) a 
a (n + 1)(n + 2) 


nN e 


This relation enables us to determine a,, a4, a,,... , successively in terms of a). Similarly, 


we can compute a3, d;, @7,..., in terms of a,. For the coefficients with even subscripts 
we have 
_ a(a + 1) ; 
2 1 ; 7 0> 
a—2\(a+ 3 a(a — 2a + 1)\(a + 3 
a, ee ( x Yi os (—1)? ( )( i )( Me 


and, in general, 


aay = (I ERD te FE Io + 3) (a+ 2n— 1) 


This can be proved by induction. For the coefficients with odd subscripts we find 


n(@ — (a — 3) + (@ = 2n + 1) (% + 2+ 4) °° (e+ 20) | 
(2n + 1)! 


Gentil = = 1) 


Therefore the series for y can be written as 


(6.37) Y = Aguy(X) + ayup(x), 
where 
(6.38) ; 
u(x) = 1 + D(-0 seme = acai 
and 
(6.39) ; 
eae +2 seacmeieaoeemacammae’ "5 anemealcaamacarate antl 


The ratio test shows that each of these series converges for |x| <1. Also, since the 
relation (6.36) is satisfied separately by the even and odd coefficients, each of u, and u, is a 
solution of the differential equation (6.35). These solutions satisfy the initial conditions 


u(0)=1, u,0)=0, u,(0)=0, u,(0)=1. 
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Since u, and uw, are independent, the general solution of the Legendre equation (6.35) over 
the open interval (—1, 1) is given by the linear combination (6.37) with arbitrary constants 
a) and a,. 
When «a is 0 or a positive even integer, say « = 2m, the series for u,(x) becomes a 
polynomial of degree 2m containing only even powers of x. Since we have 
2"m! 

a(a — 2)°+- (a — 2n + 2) = 2m(2m — 2):-+(Qm — 2n 4+ 2) = ———— 
(m — n)! 
and 

(a + 1)\(a + 3)°°+ (a 4+ 2n — 1) = (2m 4+ 1)Qm + 3)-+- (2m + 2n — 1) 


_ (2m + 2n)! m! 
2"(2m)! (m + n)! 


the formula for u,(x) in this case becomes 


_ a I? 4 (2m + 2k)! 2k 
om eer 5 m- blame bIObl | 


For example, when « = 0, 2, 4, 6 (m = 0, 1, 2, 3) the corresponding polynomials are 
u(x) = 1, 1 — 3x3, 1 — 10x? + 32x74, 1 — 21x? + 63x14 — 234x8, 


The series for u,(x) is not a polynomial when « is even because the coefficient of x?"*? is 
never zero. 

When «@ is an odd positive integer, the roles of u, and wu, are reversed; the series for 
Uu,(x) becomes a polynomial and the series for u,(x) is not a polynomial. Specifically if, 
a = 2m + | we have 


(6.41) u(x) = x + ——+—_ ane oh ae ye ee amass > sae 


(m —k)!(m+ k)!(2k + 1)! 
For example, when « = 1, 3,5 (m =0, 1, 2), the corresponding polynomials are 
u(x)=x, x— 8x8, x — Fyx8 + Ahx?, 


6.19 The Legendre polynomials 


Some of the properties of the polynomial solutions of the Legendre equation can be 
deduced directly from the differential equation or from the formulas in (6.40) and (6.41). 
Others are more easily deduced from an alternative formula for these polynomials which 
we shall now derive. 

First we shall obtain a single formula which contains (aside from constant factors) both 
the polynomials in (6.40) and (6.41). Let 


_ 1S 1 Gn = 2) nate 
(6.42) PAC) = 5, 2 rp yee es gill 
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where [n/2] denotes the greatest integer < n/2. We will show presently that this is the 
Legendre polynomial of degree n introduced in Chapter 1. When 7 is even, it is a constant 
multiple of the polynomial u,(x) in Equation (6.40); when 7 is odd, it is a constant multiple 
of the polynomial u(x) in (6.41). The first seven Legendre polynomials are given by the 
formulas 


P(x)=1, PixX)=x, P(x) = 33x? — 1), — Ps(x) = $(5x° — 3x), 
P,(x) = 4(35x* — 30x? + 3), P;(x) = 3(63x° — 70x? + 15x), 
P,(x) = Pd = 315x4 + 105x? rams >): 


Figure 6.1 shows the graphs of the first five of these functions over the interval [—1, 1]. 


y 


FicureE 6.1 Graphs of Legendre polynomials over the interval [—1, 1]. 


Now we can show that, except for scalar factors, the Legendre polynomials are those 
obtained by applying the Gram-Schmidt orthogonalization process to the sequence of 
polynomials 1, x, x?,..., with the inner product 


(f, 8) = ff (x)g(x) dx. 


+ When x is even, say n = 2m, we may replace the index of summation k in Equation (6.40) by a new index 
r, where r = m — k, we find that the sum in (6.40) is a constant multiple of P,(x). Similarly, when x is 
odd, a change of index transforms the sum in (6.41) to a constant multiple of P,(x). 
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First we note that if m 4 n the polynomials P,, and P,, are orthogonal because they are 
eigenfunctions of a symmetric operator belonging to distinct eigenvalues. Also, since P,, 
has degree n and Py) = 1, the polynomials P(x), P\(x),..., P,(x) span the same subspace 
as 1,x,...,x”. In Section 1.14, Example 2, we constructed another orthogonal set of 
polynomials yo, yi, ¥e,---, such that yo(x), y,(x),..., n(x) spans the same subspace as 
1,x,...,x" for each n. The orthogonalization theorem (Theorem 1.13) tells us that, 
except for scalar factors, there is only one set of orthogonal functions with this property. 
Hence we must have 


P(X) = CrYn(X) 


for some scalars c,,. The coefficient of x” in y,(x) is 1, so c,, is the coefficient of x” in P,,(x). 
From (6.42) we see that 


_ nt 
7 2"(n!)?- 


6.20 Rodrigues’ formula for the Legendre polynomials 
In the sum (6.42) defining P,,(x) we note that 


(2n — 2r)! ner a y2n-2r and I > ee 
(n — 2r)! dx" ri(n—r)! nit\r]’ 


where (”) is the binomial coefficient, and we write the sum in the form 


1 q” (n/2] 


2"n! dx el (7) -_ 


When [n/2] <r <n, the term x2"-?" has degree less than n, so its nth derivative is zero. 
Therefore we do not alter the sum if we allow r to run from 0 to n. This gives us 


P,(x) = 


pio) = sh = 3 vr(")s onan 


Now we recognize the sum on the right as the binomial expansion of (x? — 1)”. Therefore 
we have 


x? — . 
Py(x) = So e 
This is known as Rodrigues’ formula, in honor of Olinde Rodrigues (1794-1851), a French 
economist and reformer. 

Using Rodrigues’ formula and the differential equation, we can derive a number of 
important properties of the Legendre polynomials. Some of these properties are listed 
below. Their proofs are outlined in the next set of exercises. 

For each n > 0 we have 


P,(1) = 1. 
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Moreover, P,,(x) is the only polynomial which satisfies the Legendre equation 
(1 — x?)y” — 2xy’ + n(n + l)y = 0 
and has the value 1 when x = 1. 
For each n > 0 we have 


P,(—x) = (—1)"P,, (2). 


This shows that P, is an even function when n is even, and an odd function when n is odd. 
We have already mentioned the orthogonality relation, 


[L PCP) dx =0 if men. 


When m = n we have the norm relation 


Pal? = [_ (PQOF ax = A 


Every polynomial of degree n can be expressed as a linear combination of the Legendre 
polynomials Py, P,,...,P,. In fact, if fis a polynomial of degree n we have 


I(x) = 2 Pa) 
where = 
_ 2k+1 
2 


Cr 


[ soorcs dx. 


From the orthogonality relation it follows that 


is 2(x)P,(x) dx = 0 


for every polynomial g of degree less than n. This property can be used to prove that the 
Legendre polynomial P,, has n distinct real zeros and that they all lie in the open interval 


(—1, 1). 


6.21 Exercises 


1. The Legendre equation (6.35) with « = 0 has the polynomial solution u,(x) = 1 and a solution 
u,, not a polynomial, given by the series in Equation (6.41). 
(a) Show that the sum of the series for uw, is given by 


1 Fs ae, 
u(x) = i for |x| <1. 


(b) Verify directly that the function uz in part (a) is a solution of the Legendre equation when 
a =(Q. 
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2. Show that the function f defined by the equation 


1+x 
l1-x 


f(x) =1 — 5 log 


for |x| < 1 satisfies the Legendre equation (6.35) with « = 1. Express this function as a linear 
combination of the solutions u, and uy given in Equations (6.38) and (6.39). 
3. The Legendre equation (6.35) can be written in the form 


(x? — Dy’) — «(a + Dy =0. 


(a) If a, b, c are constants with a > b and 4c + 1 > 0, show that a differential equation of 
the type 


[(x — a)(x — by’! — cy =0 


can be transformed to a Legendre equation by a change of variable of the form x = At + B, 
with A > 0. Determine A and B in terms of a and b. 
(b) Use the method suggested in part (a) to transform the equation 


(x? — x)y” + QQx — ly’ —2y =0 
to a Legendre equation. 
4. Find two independent power-series solutions of the Hermite equation 


y” — 2xy’ + 2ay =0 


on an interval of the form (—r , r). Show that one of these solutions is a polynomial when « is a 
nonnegative integer. 
5. Find a power-series solution of the differential equation 


xy” + (3 + x3)y’ + 3x2y =0 


valid for all x. Find a second solution of the form y = x~* > a,x" valid for all x # 0. 
6. Find a power-series solution of the differential equation 


xty” + x®y’ — (ax + 2)y =0 


valid on an interval of the form (—r,r). 
7. Given two functions A and B analytic on an interval (x9 —r,X»9 +1), say 


A(x) = S aglx — x0)", BG) = S bax — 9)". 
n=0 n=0 


It can be shown that the product C(x) = A(x)B(x) is also analytic on (%) —r,Xy +17). This 
exercise shows that C has the power-series expansion 


foe) n 
C(x) = ¥ cn(x — Xp)", where Cy = > ayDn_x- 
n=0 k=0 
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(a) Use Leibniz’s rule for the ath derivative of a product to show that the nth derivative of C is 
given by 


cin) a. — (0 Al®) (x) Bi") ; 
@) = > (,)4P era) 


(b) Now use the fact that A™)(x,) = k! a, and B!-*)(x,) = (n — k)! b,_; to obtain 
C(x) =n! ¥ aybr_n- 
k=0 


Since C'! (x9) = n!c,, this proves the required formula for c,. 


In Exercises 8 through 14, P,(x) denotes the Legendre polynomial of degree n. These exercises 
outline proofs of the properties of the Legendre polynomials described in Section 6.20. 


8. (a) Use Rodrigues’ formula to show that 


1 
Prax) = 5% +" + & — DO), 
where Q,,(x) is a polynomial. 
(b) Prove that P,(1) = 1 and that P,(-—1) = (-1)”. 
(c) Prove that P,(x) is the only polynomial solution of Legendre’s equation (with « = n) 
having the value 1 when x = 1. 
9. (a) Use the differential equations satisfied by P,, and P,, to show that 
[lh — x*)(PaPin — PpPm)Y = [nt + 1) — mim + DIPpPn- 


(b) If m # n, integrate the equation in (a) from —1 to | to give an alternate proof of the 
orthogonality relation 


[7 Pa@)Pm(x) dx = 0. 
10. (a) Let f(x) = (x? — 1)”. Use integration by parts to show that 
ic fM~f™Od ax = -|" fevayfeYV@) dx 
~1 —1 ; 
Apply this formula repeatedly to deduce that the integral on the left is equal to 
2(2n)! [* (1 — x2)" de. 
0 


(b) The substitution x = cos ¢ transforms the integral J} (1 — x?)" dx to Jz’? sin?"+14 dt. Use 


the relation 
ree? 2n(2n ~2)---2 
a =n On = 12423" 


and Rodrigues’ formula to obtain 


[7 P.@OF dx = : 
a ee 2n+1— 
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11. 


1a 


13. 


14. 


15. 
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(a) Show that 
(2n)! 
Pr(x) = 2"(n nye + On(x), 


where Q,,(x) is a polynomial of degree less than n. 

(b) Express the polynomial f(x) = x‘ as a linear combination of Py, P,, Po, Ps, and Py. 

(c) Show that every polynomial f of degree n can be expressed as a linear combination of the 
Legendre polynomials Py, Py,..., Pn. 

(a) If fis a polynomial of degree n, write 


fo) => Ppa). 
k=0 


[This is possible because of Exercise 11(c).] Fora fixed m,0 < m <n, multiply both sides of 
this equation by P,,(x) and integrate from —1 to 1. Use Exercises 9(b) and 10(b) to deduce the 
relation 


2m a 1 
Cm = [. (Xx) Pm(x) dx. 


Use Exercises 9 and 11 to show that J1, 2(x)P,(x) dx = 0 for every polynomial g of degree 
less than a. 

(a) Use Rolle’s theorem to show that P, cannot have any multiple zeros in the open interval 
(—1, 1). In other words, any zeros of P,, which lie in (—1 , 1) must be simple zeros. 

(b) Assume P,, has m zeros in the interval (—1, 1). If m =0, let Q(x) = 1. If m > 1, let 


Om(x) = (% — xy) — Xg) °° (% — Xm); 


where x1, X2,..., Xm are the m zeros of P,, in(—1, 1). Show that, at each point xin (—1, 1), 
O»(x) has the same sign as P,,(x). 

(c) Use part (b), along with Exercise 13, to show that the inequality m <n leads to a con- 
tradiction. This shows that P,, has n distinct real zeros, all of which lie in the open interval 
(-—1, 1). 

(a) Show that the value of the integral {1 P,,(x)P; n+1(X) dx is independent of n. 

(b) Evaluate the integral j*, x P,(x)P,_4(x) ax. 


6.22 The method of Frobenius 


In Section 6.17 we learned how to find power-series solutions of the differential equation 


(6.43) y+ Pi)y’ + Pa(xy = 0 


in an interval about a point x, where the coefficients P,; and P, are analytic. If either P, or 


Py i 


s not analytic near x), power-series solutions valid near x) may or may not exist. For 


example, suppose we try to find a power-series solution of the differential equation 


(6.44) xy" —y —y=0 


nea 


rX) = 0. If we assume that a solution y = > a,x* exists and substitute this series in the 


differential equation we are led to the recursion formula 


_n—n-1 
a4 = 


n+l 
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Although this gives us a power series y = > a,x* which formally satisfies (6.44), the ratio 
test shows that this power series converges only for x = 0. Thus, there is no power-series 
solution of (6.44) valid in any open interval about x» = 0. This example does not violate 
Theorem 6.13 because when we put Equation (6.44) in the form (6.43) we find that the 
coefficients P, and P, are given by 


1 1 
P(x) =e and P,(x) = 2 ae 
x xX 


These functions do not have power-series expansions about the origin. The difficulty here 
is that the coefficient of y” in (6.44) has the value 0 when x = 0; in other words, the 
differential equation has a singular point at x = 0. 

A knowledge of the theory of functions of a complex variable is needed to appreciate the 
difficulties encountered in the investigation of differential equations near a singular point. 
However, some important special cases of equations with singular points can be treated by 
elementary methods. For example, suppose the differential equation in (6.43) is equivalent 
to an equation of the form 


(6.45) (x — Xo)*y" + (x — Xo)P(x)y’ + Ox)y = 0, 


where P and Q have power-series expansions in some open interval (x) —r, x) +r). In 
this case we say that x, is a regular singular point of the equation. If we divide both sides 
of (6.45) by (x — x)? the equation becomes 


y" + Fe) y+ 
X — Xo (x — Xp) 
for x # Xo. If P(xo) ¥ 0 or O(x%) ¥ O, or if O(x,) = 0 and Q’(x,) ¥ 0, either the co- 
efficient of y' or the coefficient of y will not have a power-series expansion about the point xo, 
so Theorem 6.13 will not be applicable. In 1873 the German mathematician Georg Fro- 
benius (1849-1917) developed a useful method for treating such equations. We shall 
describe the theorem of Frobenius but we shall not present its proof.+ In the next section 
we give the details of the proof for an important special case, the Bessel equation. 

Frobenius’ theorem splits into two parts, depending on the nature of the roots of the 
quadratic equation 


(6.46) t(t — 1) + P(X)t + Q(x) = 0. 


This quadratic equation is called the indicial equation of the given differential equation 
(6.45). The coefficients P(x») and Q(x) are the constant terms in the power-series ex- 
pansions of P and Q. Let «, and «, denote the roots of the indicial equation. These roots 
may be real or complex, equal or distinct. The type of solution obtained by the Frobenius 
method depends on whether or not these roots differ by an integer. 


} For a proof see E. Hille, Analysis, Vol. II, Blaisdell Publishing Co., 1966, or E. A. Coddington, An 
Introduction to Ordinary Differential Equations, Prentice-Hall, 1961. 
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THEOREM 6.14. FIRST CASE OF FROBENIUS’ THEOREM. Let a, and a, be the roots of the 
indicial equation and assume that «, — %, is not an integer. Then the differential equation 
(6.45) has two independent solutions u, and uy of the form 


(6.47) u(x) = |x — xl > a,(x — Xo)", with ag=1, 
n=0 

and 

(6.48) U(X) = |x — Xl? > 5, (x — Xo)”, with bo =1. 
n=0 


Both series converge in the interval |x — x9| <r, and the differential equation is satisfied for 
0< |x —x| <r. 


THEOREM 6.15. SECOND CASE OF FROBENIUS’ THEOREM. Let «,, &% be the roots of the 
indicial equation and assume that a, — %, = N, a nonnegative integer. Then the differential 
equation (6.45) has a solution u, of the form (6.47) and another independent solution uz, of the 
form 


(6.49) u(x) = |x — Xol"* & bal — Xo)" + C u,(x) log |x — xl, 


where bj = 1. The constant C is nonzero if N=0. If N> 0, the constant C may or may 
not be zero. As in Case 1, both series converge in the interval |x — xo| <r, and the solutions 
are valid for 0 < |x — x9| <r. 


6.23 The Bessel equation 


In this section we use the method suggested by Frobenius to solve the Bessel equation 
eye ae xy’ + (x? ae a*)y — 0, 


where « is a nonnegative constant. This equation is used in problems concerning vibrations 
of membranes, heat flow in cylinders, and propagation of electric currents in cylindrical 
conductors. Some of its solutions are known as Bessel functions. Bessel functions also arise 
in certain problems in Analytic Number Theory. The equation is named after the German 
astronomer F. W. Bessel (1784-1846), although it appeared earlier in the researches of 
Daniel Bernoulli (1732) and Euler (1764). 

The Bessel equation has the form (6.45) with x» = 0, P(x) = 1, and Q(x) = x* — @?, 
so the point x, is a regular singular point. Since P and Q are analytic on the entire real line, 
we try to find solutions of the form 


(6.50) y = |x|" a,x", 


with a) ~ 0, valid for all real x with the possible exception of x = 0. 
First we keep x > 0, so that |x|' = x’. Differentiation of (6.50) gives us 


ie. @) ie.8) 2 6) 
) Hx > ax’ x Dna Sx rt Oe’: 
n=0 n=0 


n=0 
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Similarly, we obtain 


y =x? (n+ H(n +t — 1)a,x”. 
n=0 
If L(y) = x*y" + xy’ + (x? — @)y, we find 


L(y) = x3 (n + t)\(n+t—1)a,x" + x(n + t)a,x” 
n=0 n=0 


ie.0) 0 oO io 0) 
+ x' > a,x"? — x >¥ ata,x” = xd [(n + 1)? — a7 Ja,x" + x’ Ya,x?. 
n=0 n=0 n=0 n=0 
Now we put L(y) = 0, cancel x‘, and try to determine the a, so that the coefficient of each 
power of x will vanish. For the constant term we need (t? — «*)ag = 0. Since we seek a 
solution with ad) ¥ 0, this requires that 


(6.51) 2? — og = 0. 


This is the indicial equation. Its roots « and —a are the only possible values of ¢ that can 
give us a solution of the desired type. 

Consider first the choice t = «. For this ¢ the remaining equations for determining the 
coefficients become 


(6.52) (1+ «%—@]a,=0 and [(n+a)?—e2]a,+a,.=0 


forn > 2. Since « > 0, the first of these implies that a, = 0. The second formula can 
be written as 


a a 
6.53 a, = — —_—_*2 — = —- —*' 
9) (n + a)? — o? n(n + 2a) 
SO ag = a, = a, = +++ =O. For the coefficients with even subscripts we have 
Fp fae Os I es a 8 
"22 + 2a) 21 +a)’ * 4(4 4+ 2x) 2421(1 + a2 + a)’ 
PL. Sa One ar!) en 
666 + 2a) 2831(1 + a2 + a)3 +a)’ 
and, in general, 
a (=1)"ay 


~ 2nt(1 4+ a2 +a)---(n +0) 


Therefore the choice t = a gives us the solution 


ya act (t+ Dp oe | 


The ratio test shows that the power series appearing in this formula converges for all real x. 
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In this discussion we assumed that x > 0. If x < 0 we can repeat the discussion with 
x' replaced by (—x)‘. We again find that ¢ must satisfy the equation t? — a? = 0. Taking 
t = a we then obtain the same solution, except that the outside factor x* is replaced by 
(—x)*. Therefore the function /, given by the equation 


= a S Be Es 
(6.54) fal) = a |x| f ss 2 nl (1 + 2 +a)" (n+ 5) 


is a solution of the Bessel equation valid for all real x 4 0. For those values of « for which 
F,(0) and f,"(0) exist the solution is also valid for x = 0. 

Now consider the root t = —« of the indicial equation. We obtain, in place of (6.52), the 
equations 


[(d — a)? — «Ja, = 0 and [(n — a)? — oJa, +a, .,.=0, 


which become 


(1 — 2a)a, = 0 and n(n — 2a)a, + a,» = 0. 
If 2« is not an integer these equations give us a, = 0 and 


aAn_2 
a,=— 
n(n — 2a) 
forn > 2. Since this recursion formula is the same as (6.53), with « replaced by —a, we 
are led to the solution 


(6.55) fax) = ag |x|“ f +>. maar aca) 


n=1 


valid for all real x 4 0. 

The solution f_, was obtained under the hypothesis that 2« is not a positive integer. 
However, the series for f_, is meaningful even if 2« is a positive integer, so long as « is 
not a positive integer. It can be verified that f_, satisfies the Bessel equation for such «. 
Therefore, for each a > 0 we have the series solution f,, given by Equation (6.54); and 
if x is not a nonnegative integer we have found another solution /_, given by Equation (6.55). 
The two solutions f/, and f_, are independent, since one of them —oo as x — 0, and the 
other does not. Next we shall simplify the form of the solutions. To do this we need some 
properties of Euler’s gamma function, and we digress briefly to recall these properties. 


For each real s > 0 we define I'(s) by the improper integral 


I(s) = I. tre dt. 
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This integral converges if s > 0 and diverges if s < 0. Integration by parts leads to the 
functional equation 


(6.56) Nis+)=sTFQ). 
This implies that 


Nis +2)=(8+ DP 4+ 1) =(54+ Is T(s), 


T(s + 3) = (s + 2)M(s + 2) = (s +2) (s+ Ds Ts), 
and, in general, 


(6.57) T(s +n) =(st+n—1)°-:: (s+ 1s T(s) 


for every positive integer n. Since (1) = J et dt = 1, when we put s = 1 in (6.57) we 
find 
Tin+1)=n!. 


Thus, the gamma function is an extension of the factorial function from integers to positive 
real numbers. 

The functional equation (6.56) can be used to extend the definition of I'(s) to negative 
values of s that are not integers. We write (6.56) in the form 


(6.58) [(s) = “E+ 


The right-hand member is meaningful ifs + 1 > 0ands #0. Therefore, we can use this 
equation to define I(s) if —1 <<.s <0. The right-hand member of (6.58) is now meaning- 
fulifs+2>0,s +4 —1,s #0, and we can use this equation to define ['(s) for —2 < 
s < —1. Continuing in this manner, we can extend the definition of I‘(s) by induction to 
every open interval of the form —n < s < —n + 1, where n is a positive integer. The 
functional equation (6.56) and its extension in (6.57) are now valid for all real s for which 
both sides are meaningful. 


We return now to the discussion of the Bessel equation. The series for f, in Equation 
(6.54) contains the product (1 + «) (2 + «):::(" + a). We can express this product in 
terms of the gamma function by taking s = 1 + « in (6.57). This gives us 


(in + 1 + aw) 


(+ a)2+a)':‘(W+a)= Td +0 


Therefore, if we choose ay = 2-*/[\(1 + «) in Equation (6.54) and denote the resulting 
function f,(x) by J,(x) when x > 0, the solution for x > 0 can be written as 


wie 10)= JS ae 


=0 
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The function J, defined by this equation for x > 0 and « > 0 1s called the Bessel function 
of the first kind of order «. When « is a nonnegative integer, say « = p, the Bessel function 
J, is given by the power Series 


ee) 


J (x) = > or (p= 0, 1,224). 


n=0 


This is also a solution of the Bessel equation for x < 0. Extensive tables of Bessel functions 
have been constructed. The graphs of the two functions J, and J, are shown in Figure 6.2. 


- 


FIGURE 6.2 Graphs of the Bessel functions Jy and J,. 


We can define a new function J_, by replacing « by —« in Equation (6.59), if « is such 
that (mn + 1 — a) is meaningful; that is, if « is not a positive integer. Therefore, if x > 0 
anda >0,«+41,2,3,..., we define 


10) () Done cal) 


Taking s = 1 — « in (6.57) we obtain 
Ra+l—-a=(U—«)Q—24):°-™m—a)Td —«) 


and we see that the series for J_,(x) is the same as that for f_,(x) in Equation (6.55) with 
a, = 27/T(l — «), x > 0. Therefore, if « is not a positive integer, J_, is a solution of 
the Bessel equation for x > 0. 

If « is not an integer, the two solutions J,(x) and J_,(x) are linearly independent on the 
positive real axis (since their ratio is not constant) and the general solution of the Bessel 
equation for x > 0 is 

y = CJ, (x) + CoJ_,(x). 


If « is a nonnegative integer, say « = p, we have found only the solution J, and its con- 
stant multiples valid for x > 0. Another solution, independent of this one, can be found 
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by the method described in Exercise 4 of Section 6.16. This states that if u, is a solution of 
y” + Piy’ + Poy = 0 that never vanishes on an interval J, a second solution u, independent 
of u, is given by the integral 
2A | 
x) = u,(x 
uses) = my(x) [Oat 


where Q(x) = e~!P1)az_ For the Bessel equation we have P,(x) = 1/x, so Q(x) = 1/x 
and a second solution u, is given by the formula 


(6.60) ux) = J(x) | brig t, 


if c and x lie in an interval Jin which J, does not vanish. 
This second solution can be put in other forms. For example, from Equation (6.59) we 
may write 


Es ek 
(OP te f2P g,(t) 5) 


where g,(0) 4 0. In the interval / the function g, has a power-series expansion 
Bx) = 2 Ant 
which could be determined by equating coefficients in the identity g,(¢) [/,()])? = 7". If 


we assume the existence of such an expansion, the integrand in (6.60) takes the form 


°,2) 


1 1 
= A,t”. 
t{J,,(t)]° p2Pti > n 


n=0 


Integrating this formula term by term from c to x we obtain a logarithmic term A, log x 
(from the power t~) plus a series of the form x-?? >} B,x”. Therefore Equation (6.60) 
takes the form 


u(x) = Ao,J,(x) log x + J,(x)x-?? > Bx”. 
n=0 


It can be shown that the coefficient A,, 4 0. If we multiply u(x) by 1/A,, the resulting 
solution is denoted by K,(x) and has the form 


K,(x) = J,(x) log x + x-? > C,x”. 
n=0 


This is the form of the solution promised by the second case of Frobenius’ theorem. 
Having arrived at this formula, we can verify that a solution of this form actually exists 

by substituting the right-hand member in the Bessel equation and determining the co- 

efficients C,, so as to satisfy the equation. The details of this calculation are lengthy and 
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will be omitted. The final result can be expressed as 


cp sins YE)” Sean M EF 16S tes oh 


where Ay = Oandh, = 1+4+:°:-+1/nforn>1. The series on the right converges 
for all real x. The function K, defined for x > 0 by this formula is called the Bessel function 
of the second kind of order p. Since K, is not a constant multiple of J, , the general solution 
of the Bessel equation in this case for x > 0 is 


y = ¢J,(x) + ¢.K,(x). 


Further properties of the Bessel functions are discussed in the next set of exercises. 


6.24 Exercises 


1. (a) Let f be any solution of the Bessel equation of order « and let g(x) = xf (x) forx > 0. 
Show that g satisfies the differential equation 


; | — 4a! 
yr ar ee y=0. 


(b) When 4a? = 1 the differential equation in (a) becomes y” + y = 0; its general solution 
is y = Acosx + Bsinx. Use this information and the equationf I'(3) = V = to show that, 


forx >0, 
2\4 | 2 \4 
Ji(x) = (=) sin x and J_i(x) = (=) COS x. 


(c) Deduce the formulas in part (b) directly from the series for J:,(x) and J_1,(x). 
2. Use the series representation for Bessel functions to show that 


d 
(a) = (eV ,(x)) = xa), 


d 
(0) (V0) = Vga). 
3. Let F(x) = xJ,(x) and G,(x) = x~*J,(x) for x > 0. Note that each positive zero of J, is a 
zero of F, and is also a zero of G,. Use Rolle’s theorem and Exercise 2 to prove that the posi- 


tive zeros of J, and J,,, interlace. That is, there is a zero of J, between each pair of positive 
zeros Of J,44, and a zero of J,,, between each pair of positive zeros of J,. (See Figure 6.2.) 


} The change of variable t = u? gives us 
ie @) 00 —_ 
TQ) = i t—l4e-tdt = a e-uw du = Var. 
0+ 0 


(See Exercise 16 of Section 11.28 for a proof that 2 fi e-uw* du = Vz.) 


10. 


11. 
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. (a) From the relations in Exercise 2 deduce the recurrence relations 


a a 
- J Ax) + JU(x) = J,1() and : JAX) — JE) = Iya). 
(b) Use the relations in part (a) to deduce the formulas 


2 
Jy) + Ju) = — JAx) and Ig g(x) — gag (x) = N(x). 


. Use Exercise 1(b) and a suitable recurrence formula to show that 


2 \4/sin x 
Jsg(x) = —S Ee nee . 


TX 


Find a similar formula for J_3,(x). Note: J,(x) is an elementary function for every « which is 
half an odd integer. 


. Prove that 
: < (J2(x) + J2,4(@x)) = = J3(x) = —_ Jav1(X) 
and r 
Fe Sao) = xUE0) — Jz). 
. (a) Use the identities in Exercise 6 to show that 


J2(x) + 2 > J2(x) = |] and >) (2n + I)Jn(X)Jniy(x) = ox. 


n=1 n=0 


(b) From part (a), deduce that |Jj(x)| < 1 and |J,(x)| < LVv2 forn =1,2,3,..., and all 
x>0. 


. Let g,(x) = xf, (ax?) for x > 0, where a and d are nonzero constants. Show that g, satisfies 


the differential equation 
x2y” + (a2b?x® +4 — ab*%)y = 0 


if, and only if, f, is a solution of the Bessel equation of order «. 


. Use Exercise 8 to express the general solution of each of the following differential equations in 


terms of Bessel functions for x > 0. 

(a) y° +xy =0. (c) y" +x™y =0. 

(b) y” + x*y =0. (d) x2y" + (xt + Dy =0. 

Generalize Exercise 8 when f, and g, are related by the equation g,(x) = x‘f,(ax°) for x > 0. 
Then find the general solution of each of the following equations in terms of Bessel functions 
for x > 0. 

(a) xy” + 6y’ +y =0. (c) xy” + 6y’ + xty =0. 

(b) xy” + 6y’ + xy =0. (d) x?y” — xy’ +(x + Dy =0. 

A Bessel function identity exists of the form 


J2(x) — Jo(x) = a(x), 


where a and ¢c are constants. Determine a and c. 
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12. Find a power series solution of the differential equation xy” + y’ + y = 0 convergent for 
—0o <x < +0. Show that for x > 0 it can be expressed in terms of a Bessel function. 
13. Consider a linear second-order differential equation of the form 


x*A(x)y” + xP(x)y’ + O(x)y =0, 


where A(x), P(x), and Q(x) have power series expansions, 
A(x) = Yayx®, Px) = SY p,px*, OW) => qx", 
k=0 k=0 k=0 


with ay # 0, each convergent in an open interval (—r,r). If the differential equation has a 
series solution of the form 
y =xt dc,x", 
n=0 
valid for 0 < x <r, show that ¢ satisfies a quadratic equation of the form #? + bt +c =0, 
and determine b and c in terms of coefficients of the series for A(x), P(x), and Q(x). 
14. Consider a special case of Exercise 13 in which A(x) = 1 — x, P(x) = $, and Q(x) = —jx. 
Find a series solution with ¢ not an integer. 
15. The differential equation 2x*y” + (x? — x)y’ + y =0 has two independent solutions of the 
form 
[e.) 
y=! > Cox 


n=0 


valid for x > 0. Determine these solutions. 

16. The nonlinear differential equation y” + y + ay? = Ois only “mildly” nonlinear if « is a small 
nonzero constant. Assume there is a solution which can be expressed as a power series in « 
of the form 


y=) u,(x)«" (valid in some interval 0 < « <r) 


3 
IMs 


and that this solution satisfies the initial conditions y = 1 and y’ = 0 when x = 0. To con- 
form with these initial conditions, we try to choose the coefficients u,,(x)so that u4)(0) = 1, 
u,(0) = 0 and u,,(0) = u,(0) = 0 for n > 1. Substitute this series in the differential equation, 
equate suitable powers of « and thereby determine u(x) and u(x). 


7 
SYSTEMS OF DIFFERENTIAL EQUATIONS 


7.1 Introduction 


Although the study of differential equations began in the 17th century, it was not until 
the 19th century that mathematicians realized that relatively few differential equations could 
be solved by elementary means. The work of Cauchy, Liouville, and others showed the 
importance of establishing general theorems to guarantee the existence of solutions to 
certain specific classes of differential equations. Chapter 6 illustrated the use of an existence- 
uniqueness theorem in the study of linear differential equations. This chapter is concerned 
with a proof of this theorem and related topics. 

Existence theory for differential equations of higher order can be reduced to the first- 
order case by the introduction of systems of equations. For example, the second-order 
equation 


(7.1) y’' + 2ty —y=e 


can be transformed to a system of two first-order equations by introducing two unknown 
functions y, and y,, where 
ss ieee a ye = yy. 


Then we have y, = y, = y”, so (7.1) can be written as a system of two first-order equations: 


y=y 
(7.2) as 
Yo = yy — 2ty, + e'. 


We cannot solve the equations separately by the methods of Chapter 6 because each of them 
involves two unknown functions. 


In this chapter we consider systems consisting of n linear differential equations of first 
order involving n unknown functions y,,...,),- These systems have the form 
Yi = Puy + Prove Fo + Pin + (2) 
(7.3) 


Vn = Pri) f Pnolt) ye + ot te Pan) Yn + q(t). 
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The functions p,, and q; which appear in (7.3) are considered as given functions defined on 
a given interval J. The functions y,,..., y, are unknown functions to be determined. 
Systems of this type are called first-order linear systems. In general, each equation in the 
system involves more than one unknown function so the equations cannot be solved 
separately. 

A linear differential equation of order n can always be transformed to a linear system. 
Suppose the given nth order equation is 


(7.4) y™) + ayy") foeeey any = R(t), 


where the coefficients a; are given functions. To transform this to a system we write y,; = y 
and introduce a new unknown function for each of the successive derivatives of y. That is, 
we put 


W=y), Yo= Vis V3 = Var+++9 Vn = Vn 


and rewrite (7.4) as the system 


Vi = ye 
yo = Vs 
(7.5) 
Vn-1 = Yn 
Vn = —AnyYi — An_-1Y2—- °° * — An F R(t). 


The discussion of systems may be simplified considerably by the use of vector and matrix 
notation. Consider the general system (7.3) and introduce vector-valued functions Y = 
O1>-++sVn)> OQ = (Gi,-++59n), and a matrix-valued function P = [p,,], defined by the 
equations 


Y(t) = (ilt), ++» Ynlt)), Ot) = (Milt), - snl), = P(t) = [piilt)] 


for each tinJ. We regard the vectors asm X 1 column matrices and write the system (7.3) 
in the simpler form 


(7.6) Y’ = P(t)Y + O(t). 


For example, in system (7.2) we have 


vei a em 
a y, (t) = i “ah O( ) — a 
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In system (7.5) we have 


0 0 
= 0 0 1 0 0 
Jy2 : ; ; . 
y=| oj, POs} s+ |, Q= 
0 0 0 one l 0 
Vn 
—A, An. TAn-2 °°° “Qh | R(t) 


An initial-value problem for system (7.6) is to find a vector-valued function Y which 
satisfies (7.6) and which also satisfies an initial condition of the form Y(a) = B, where 
aéeJand B= (b,,...,5,) is a given n-dimensional vector. 

In the case n = 1 (the scalar case) we know from Theorem 6.1 that, if P and Q are 
continuous on J, all solutions of (7.6) are given by the explicit formula 


(7.7) ¥(x) = e#*y(a) + e4 [* eAMQ(N dt, 


where A(x) = f* P(t) dt, and a is any point in J. We will show that this formula can be 
suitably generalized for systems, that is, when P(t) is ann X n matrix function and Q(t) 
is an n-dimensional vector function. To do this we must assign a meaning to integrals of 
matrices and to exponentials of matrices. Therefore, we digress briefly to discuss the 
calculus of matrix functions. 


7.2 Calculus of matrix functions 


The generalization of the concepts of integral and derivative for matrix functions is 
straightforward. If P(t) = [p,,(t)], we define the integral |? P(t) dt by the equation 


.° P(t) dt = [’ D; it) an ; 


That is, the integral of matrix P(t) is the matrix obtained by integrating each entry of P(r), 
assuming of course, that each entry is integrable on [a, b]. The reader can verify that the 
linearity property for integrals generalizes to matrix functions. 

Continuity and differentiability of matrix functions are also defined in terms of the 
entries. We say that a matrix function P = [p,,] is continuous at ¢ if each entry p;; is 
continuous at ¢. The derivative P’ is defined by differentiating each entry, 


P(t) = [pi], 


whenever all derivatives p;,(t) exist. It is easy to verify the basic differentiation rules for 
sums and products. For example, if P and Q are differentiable matrix functions, we have 


(P+ OQ) =P'+Q 
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if P and Q are of the same size, and we also have 
(PQ) = PQ’ + PQ 


if the product PQ is defined. The chain rule also holds. That is, if F(t) = P[g(t)], where P 
is a differentiable matrix function and g is a differentiable scalar function, then F’(t) = 
g'(t)P’[g(t)]. The zero-derivative theorem, and the first and second fundamental theorems 
of calculus are also valid for matrix functions. Proofs of these properties are requested in 
the next set of exercises. 

The definition of the exponential of a matrix is not so simple and requires further 
preparation. This is discussed in the next section. 


7.3 Infinite series of matrices. Norms of matrices 


Let A = [a,,;] be an n xX n matrix of real or complex entries. We wish to define the 
exponential e“ in such a way that it possesses some of the fundamental properties of the 
ordinary real or complex-valued exponential. In particular, we shall require the law of 
exponents in the form 


(7.8) e'4es4 — eltt+S)A for all real s and t, 

and the relation 

(7.9) e89 = 1, 

where O and J are then x n zero and identity matrices, respectively. It might seem natural 


to define e-t to be the matrix [e7’]. However, this is unacceptable since it satisfies neither of 
properties (7.8) or (7.9). Instead, we shall define e4 by means of a power series expansion, 


fo @) 
=> 


AX 

imo k! 

We know that this formula holds if A is a real or complex number, and we will prove that 
it implies properties (7.8) and (7.9) if A is a matrix. Before we can do this we need to 
explain what is meant by a convergent series of matrices. 


DEFINITION OF CONVERGENT SERIES OF MATRICES. Given an infinite sequence of m X n 

ices {C;,} wh l if bers, denote the ij-ent C, by cl. I 

matrices {C,,$ whose entries are real or complex numbers, denote the ij-entry of C, by c;*’. If 
all mn series 


(7.10) 2 (i=1,...,m;j=1,...,n) 
k= 


are convergent, then we say the series of matrices >.°_, C;, is convergent, and its sum is defined 
to be the m x n matrix whose ij-entry is the series in (7.10). 


Exercises 195 


A simple and useful test for convergence of a series of matrices can be given in terms of the 
norm of a matrix, a generalization of the absolute value of a number. 


DEFINITION OF NORM OF A MATRIX. Jf A = [a;,;] is an m X n matrix of real or complex 
entries, the norm of A, denoted by ||A\|, is defined to be the nonnegative number given by the 
formula 


(7.11) |All = ») ¥ lal 


i=1 j=1 


In other words, the norm of A is the sum of the absolute values of all its entries. There 
are other definitions of norms that are sometimes used, but we have chosen this one because 
of the ease with which we can prove the following properties. 


THEOREM 7.1. FUNDAMENTAL PROPERTIES OF NORMS. For rectangular matrices A and B, 
and all real or complex scalars c we have 


|A + Bll < |All + |B, JAB < [Al PBI, IeAll = lel Al. 


Proof. We prove only the result for ||AB||, assuming that A is m xX n and Bisn x p. 
The proofs of the others are simpler and are left as exercises. 
Writing A = [a;,], B = [b,;], we have AB = [7_, ai,b,;], so from (7.11) we obtain 


m y#) n 
|| AB|| = 2 2 indi 


Note that in the special case B = A the inequality for ||AB|| becomes ||A?]| < ||A|l?. 
By induction we also have 


mn Pp mn 
<> Dail Doel SX Dlaael Bi = |All (Bl. 
i=1k=1 j=1 i=1k=1 


A‘ < | Al® for k=1,2,3,.... 


These inequalities will be useful in the discussion of the exponential matrix. 
The next theorem gives a useful sufficient condition for convergence of a series of matrices. 


THEOREM 7.2. TEST FOR CONVERGENCE OF A MATRIX SERIES. Jf {C;,} is a sequence ofm X n 
matrices such that > ||C,,|| converges, then the matrix series bs _ GC, also converges. 


Proof. Let the ij-entry of C, be denoted by c‘*’. Since |ci| < ||C,l|, convergence of 
>2., ||Cyl| implies absolute convergence of each series >,” , c\#). Hence each series >? , cj) 
is convergent, so the matrix series > , C, is convergent. 


7.4 Exercises 


1. Verify that the linearity property of integrals also holds for integrals of matrix functions. 
2. Verify each of the following differentiation rules for matrix functions, assuming P and Q are 
differentiable. In (a), P and Q must be of the same size so that P + Q is meaningful. In 
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(b) and (d) they need not be of the same size provided the products are meaningful. In (c) 
and (d), Q is assumed to be nonsingular. 

(a) (P+ QO) =P'+Q. (c) (Q") = -Q70'0"'. 

(b) (PQ)’ = PQ’ + P’Q. (d) (PQ) = —PQ'10'01+P’Q". 


. (a) Let P bea differentiable matrix function. Prove that the derivatives of P? and P® are given 


by the formulas 
(P?)’ = PP’ +P'P,  (P*) = P?P’ + PP’P + P’P?. 


(b) Guess a general formula for the derivative of P* and prove it by induction. 


. Let P be a differentiable matrix function and let g be a differentiable scalar function whose 


range is a subset of the domain of P. Define the composite function F(t) = P[g(t)] and prove 
the chain rule, F’(t) = g'()P'[g(0)]. 


. Prove the zero-derivative theorem for matrix functions: If P’(t) = O for every t in an open 


interval (a, b), then the matrix function P is constant on (a, b). 


. State and prove generalizations of the first and second fundamental theorems of calculus for 


matrix functions. 


. State and prove a formula for integration by parts in which the integrands are matrix functions. 
. Prove the following properties of matrix norms: 


|A+ Bl < Al + 1B, = leAll = lel Al. 


. Ifa matrix function P is integrable on an interval [a, b] prove that 


i #Y) ar| < J? IP@ll de. 


. Let D be ann x n diagonal matrix, say D = diag (4,,..., 4,). Prove that the matrix series 


do D*/k! converges and is also a diagonal matrix, 


>a = diag (e41,..., e%). 
k=0 


(The term corresponding to k = 0 is understood to be the identity matrix I.) 
Let D be ann X n diagonal matrix, D = diag (A,,...,4,). If the matrix series pie (cuD” 
converges, prove that 


y c,D* = diag (Scat cura Ss ex) : 
k=0 k=0 


k=0 


Assume that the matrix series byes , Cz, converges, where each C, isan xX mmatrix. Prove that 
the matrix series } °°, (AC;,B) also converges and that its sum is the matrix 


4($ c)2. 


Here A and B are matrices such that the products AC;,B are meaningful. 
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7.5 The exponential matrix 
Using Theorem 7.2 it is easy to prove that the matrix series 


A*® 


(7.12) ; 


rs 


converges for every square matrix A with real or complex entries. (The term corresponding 
to k = 0 is understood to be the identity matrix 7.) The norm of each term satisfies the 


inequality 
li 


k!} 


|| A||* 
a dele 


Since the series > a*/k! converges for every real a, Theorem 7.2 implies that the series in 
(7.12) converges for every square matrix A. 


DEFINITION OF THE EXPONENTIAL MATRIX. For any n X n matrix A with real or complex 
entries we define the exponential e4 to be the n X n matrix given by the convergent series in 


(7.12). That is, 
pipet 


Note that this definition implies e? = J, where O is the zero matrix. Further properties 
of the exponential will be developed with the help of differential equations. 


eA 


7.6 The differential equation satisfied by e'4 


Let ¢ be a real number, let A be ann xX n matrix, and let E(t) be then x n matrix given by 


E(t) = e'4 


We shall keep A fixed and study this matrix as a function of t. First we obtain a differential 
equation satisfied by E 


THEOREM 7.3. For every real t the matrix function E defined by E(t) = e'4 satisfies the 
matrix differential equation 
E’(t) = E(t)A = AE(t). 


Proof. From the definition of the exponential matrix we have 


E(t) = -t- _ bigs te A® . 
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Let c‘*) denote the ij-entry of A*. Then the ij-entry of t*A*/k! is t*c\?’/k!. Hence, from the 
definition of a matrix series, we have 


a th Ae —* ph 
(7.13) a — 
Z kt [Sok 


Each entry on the right of (7.13) is a power series in t, convergent for all t. Therefore its 
derivative exists for all ¢ and is given by the differentiated series 


o k-1 Oe ght 
> Lae > Leer 
! 29 ! (e) 7 
k! cat: 


k=1 


This shows that the derivative E’(t) exists and is given by the matrix series 


2 pe AEt oe te AK 
E'(t) = = (5 Ay, = E(t)A. 


k=0 


In the last equation we used the property A*+? = A*A. Since A commutes with A* we could 
also have written A*t! = AA* to obtain the relation E’(t) = AE(t). This completes the 
proof. 


Note: The foregoing proof also shows that A commutes with e’4. 


7.7 Uniqueness theorem for the matrix differential equation F’(t) = AF(t) 


In this section we prove a uniqueness theorem which characterizes all solutions of the 
matrix differential equation F’(t) = AF(t). The proof makes use of the following theorem. 


THEOREM 7.4. NONSINGULARITY OF e4, For anyn X n matrix A and any scalar t we have 
(7.14) el4et4 = |, 
Hence e'4 is nonsingular, and its inverse is e*4, 
Proof. Let F be the matrix function defined for all real ¢ by the equation 
F(t) = e4e*4, 


We shall prove that F(t) is the identity matrix 7 by showing that the derivative F’(t) is the 
zero matrix. Differentiating F as a product, using the result of Theorem 7.3, we find 


F'(t) ea e'4(et4y a8 (e'4)'et4 os e'4(— Ae *4) + Aet4e—tA 


os — Ae4e-4 + Ae’4e—'4 = O, 


since A commutes with e’4. Therefore, by the zero-derivative theorem, F is a constant 
matrix. But F(0) = e94e94 = J, so F(t) = I for all t. This proves (7.14). 
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THEOREM 7.5. UNIQUENESS THEOREM. Let A and B be given n X n constant matrices. 
Then the only n X n matrix function F satisfying the initial-value problem 


F(t) = AF(t), F(0)=8B 
for—w<t<+ois 


(7.15) F(t) = eB. 


Proof. First we note that e‘4B is a solution. Now let F be any solution and consider the 
matrix function 


G(t) = e4F(t). 
Differentiating this product we obtain 
G'(t) = e4F'(t) — Ae“4F(t) = e4AF(t) — e 4A F(t) = O. 
Therefore G(t) is a constant matrix, 
G(t) = G(0O) = F(O) = B. 


In other words, e~‘4F(t) = B. Multiplying by e‘4 and using (7.14) we obtain (7.15). 


Note: The same type of proof shows that F(t) = Be'4 is the only solution of the 
initial-value problem 


F'(t) = FMA, F(O) = B. 


7.8 The law of exponents for exponential matrices 


The law of exponents e4e? = e4+¥ is not always true for matrix exponentials. A counter 
example is given in Exercise 13 of Section 7.12. However, it is not difficult to prove that 
the formula is true for matrices A and B which commute. 


THEOREM 7.6. Let A and B be twon X n matrices which commute, AB = BA. Then we 
have 


(7.16) ett B — eAteB 


Proof. From the equation AB = BA we find that 
A?B = A(BA) = (AB)A = (BA)A = BA?, 
so B commutes with A®. By induction, B commutes with every power of A. By writing 
e'4 as a power Series we find that B also commutes with e’4 for every real ¢. 


Now let F be the matrix function defined by the equation 


F(t) = ef(4+B) — ot4e'B 
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Differentiating F(t) and using the fact that B commutes with e’4 we find 
F'(t) = (A a B)e\4+8) _ Ae4 eB _ eABolB 
= (A + B)e4+8) _ (4 + B)e’4e'? = (A + B)F(t). 


By the uniqueness theorem we have 


F(t) _ et(4+ BF) ; 
But F(0) = O, so F(t) = O for all t. Hence 


QHATB) tA gtB 


When t = 1 we obtain (7.16). 


EXAMPLE. The matrices sA and tA commute for all scalars s and t. Hence we have 


erAetA = elsttAa 


7.9 Existence and uniqueness theorems for homogeneous linear systems with constant 
coefficients 


The vector differential equation Y’(t) = A Y(t), where A is ann X nconstant matrix and 
Y is an n-dimensional vector function (regarded as an n X 1 column matrix) is called a 
homogeneous linear system with constant coefficients. We shall use the exponential matrix 
to give an explicit formula for the solution of such a system. 


THEOREM 7.7. Let A be a givenn X n constant matrix and let B be a given n-dimensional 
vector. Then the initial-value problem 


(7.17) Y'(t)=AY(t), Y(0) = B, 


has a unique solution on the interval —o <t < +0. This solution is given by the formula 


(7.18) Y(t) = 4B. 
More generally, the unique solution of the initial value problem 


Y'(t)= AY(‘), Y(a) = B, 
is Y(t) = e-4B, 


Proof. Differentiation of (7.18) gives us Y(t) = Ae’4B = AY(t). Since Y(O) = B, 
this is a solution of the initial-value problem (7.17). 

To prove that it is the only solution we argue as in the proof of Theorem 7.5. Let Z(t) 
be another vector function satisfying Z’(t) = AZ(t) with Z(0) = B, and let G(t) = e'4Z(t). 
Then we easily verify that G’(t)=O, so G(t) = G(O) = Z(0) = B. In other words, 
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e4Z(t) = B, so Z(t) = e’4B = Y(t). The more general case with initial value Y(a) = B 
is treated in exactly the same way. 


7.10 The problem of calculating e'4 


Although Theorem 7.7 gives an explicit formula for the solution of a homogeneous 
system with constant coefficients, there still remains the problem of actually computing 
the exponential matrix e‘4. If we were to calculate e’4 directly from the series definition we 
would have to compute all the powers A* fork = 0,1,2,..., and then compute the sum 
of each series >,” , t*c\*/k! , where c\*) is the ij-entry of A*. In general this is a hopeless 
task unless A is a matrix whose powers may be readily calculated. For example, if A is a 
diagonal matrix, say 

A = diag (A,,...,4,), 


then every power of A is also a diagonal matrix, in fact, 
A*® = diag (Av, ..., A*). 


Therefore in this case e‘4 is a diagonal matrix given by 


ic) t* 00 t* 
eA en diag ( kt Ar pele se >= in) = diag (e%41, ee etAn) | 
k=0 ~° k : 


Another easy case to handle is when A is a matrix which can be diagonalized. For 
example, if there is a nonsingular matrix C such that C~'AC is a diagonal matrix, say 
C-1AC = D, then we have A = CDC"!, from which we find 


A? = (CDC-)(CDC-) = CD*C-, 


and, more generally, 


A* = CD‘C"!. 
Therefore in this case we have 
tA __ Sides ae ko—-l __ | tDr—1 
pe 7 = 2,5, 0RtC - (5 . Jc CPC, 


Here the difficulty lies in determining C and its inverse. Once these are known, e‘4 is easily 
calculated. Of course, not every matrix can be diagonalized so the usefulness of the 
foregoing remarks 1s limited. 


5 4 
EXAMPLE 1. Calculate e‘4 for the 2 x 2 matrix A = i i 


Solution. This matrix has distinct eigenvalues A, = 6, A, = 1, so there is a nonsingular 


a 
matrix C= 


b 6 0 
such that C-1AC = D, where D = diag (A,, A.) = , : . To 
c 
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determine C we can write AC = CD, or 
5 4][a b a b|{f6 0 
1 2\le d} |e djlo 1} 
Multiplying the matrices, we find that this equation is satisfied for any scalars a, b, c, d with 
a=4c,b= —d. Taking c = d = 1 we choose 


; * _ a | 1 : 

~ op. a? ~ 5L-1 4] ° 
Therefore 
1/4 -1 e OQ 1 1 
eA —_ Ce’PC7} aes 

5.1 1JLO ejJL—-1 4 
|" || e®! fi : +e 4e% — fl 

«5h tlL—-e 4c] S5Le&t—e ef + 4et | 

EXAMPLE 2. Solve the linear system 


yi = Sy, + 4ye 
Ye= yi + 2yz, 


subject to the initial conditions y,(0) = 2, y,(0) = 3. 


Solution. In matrix form the system can be written as 


Ys AY Y(O ; h A _< 
(t)= AY(t), o=|)) where -|) | 


By Theorem 7.7 the solution is Y(t) = e’4 Y(0). Using the matrix e’4 calculated in Example 
1 we find 
. i +e 4e% — I A 
Vo «5 eft — et et 4 Aot || 3 


yy = 4c — et, =—s vn = 0? + Det” 


from which we obtain 


There are many methods known for calculating e4 when A cannot be diagonalized. 
Most of these methods are rather complicated and require preliminary matrix transforma- 
tions, the nature of which depends on the multiplicities of the eigenvalues of A. In a later 
section we shall discuss a practical and straightforward method for calculating e’4 which 
can be used whether or not A can be diagonalized. It is valid for a// matrices A and requires 
no preliminary transformations of any kind. This method was developed by E. J. Putzer 
in a paper in the American Mathematical Monthly, Vol. 73 (1966), pp. 2-7. It is based on a 
famous theorem attributed to Arthur Cayley (1821-1895) and William Rowan Hamilton 
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(1805-1865) which states that every square matrix satisfies its characteristic equation. 


First we shall prove the Cayley-Hamilton theorem and then we shall use it to obtain 
Putzer’s formulas for calculating e*4. 


7.11 The Cayley-Hamilton theorem 


THEOREM 7.8. CAYLEY-HAMILTON THEOREM. Let A be ann xX n matrix and let 
(7.19) (A) = det AI — A) = A" +e, AI +--+ + A+ Co 
be its characteristic polynomial. Then f(A) = O. In other words, A satisfies the equation 
(7.20) A” + €,j;A"* 2 ** + GA + Col = 0. 


Proof. The proof is based on Theorem 3.12 which states that for any square matrix A 
we have 


(7.21) A (cof A)* = (det A)I. 


We apply this formula with A replaced by AJ — A. Since det (AJ — A) = f(A), Equation 
(7.21) becomes 


(7.22) (AI — A)fcof (AI — A)} = f(A). 


This equation is valid for all real A. The idea of the proof is to show that it is also valid 
when A is replaced by A. 

The entries of the matrix cof (AJ — A) are the cofactors of AI — A. Except for a factor 
+1 , each such cofactor is the determinant of a minor of AJ — A of ordern — 1. Therefore 
each entry of cof (AJ — A), and hence of {cof (AJ — A)}‘, is a polynomial in A of degree 
<n —1. Therefore 


{cof (AI — A)}’ = > 7B, 


k=0 


where each coefficient B, is an n X nm matrix with scalar entries. Using this in (7.22) we 
obtain the relation 


(7.23) (AI — A) > #B, = f(DI 


which can be rewritten in the form 


n—1 n—1 
(7.24) A"Bn-1 + >, A By_1 — AB,) — ABy = Al + > Ac, 1 + col. 
k=1 k=1 
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At this stage we equate coefficients of like powers of A in (7.24) to obtain the equations 
Bi = I 
Br» — AB, 1 = Cyl 

(7.25) 

Bo = AB, = cl 

—AB, = Col. 

Equating coefficients is permissible because (7.24) is equivalent to n® scalar equations, in 
each of which we may equate coefficients of like powers of A. Now we multiply the equa- 


tions in (7.25) in succession by A”, A”!,..., A, J and add the results. The terms on the 
left cancel and we obtain 


O=A"+ec, ,A™!'4+°°:':+¢4A4+ Cl. 


This proves the Cayley-Hamilton theorem. 


Note: Hamilton proved the theorem in 1853 for a special class of matrices. A few 
years later, Cayley announced that the theorem is true for all matrices, but gave no proof. 


5 4 0 
EXAMPLE. The matrix A =|1 2 0] has characteristic polynomial 
L2 2 


f(A) = (A — IQA — 2)(A — 6) = A — 92? + 204 — 12. 
The Cayley-Hamilton theorem states that A satisfies the equation 
(7.26) A® — 9A* + 20A — 121 = 0. 


This equation can be used to express A® and all higher powers of A in terms of J, A, and 
A®, For example, we have 


A® = 9A* — 20A + 12/, 
A* = 9A® — 20A? + 12A = 9(9A? — 20A + 127) — 20A? + 124A 
= 61A? — 1684 + 108/. 


It can also be used to express A! as a polynomial in A. From (7.26) we write 
A(A? — 9A + 207) = 121, and we obtain 


A = 3,(A2 — 9A + 201). 
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7.12 Exercises 


In each of Exercises 1 through 4, (a) express A~!, A? and all higher powers of A as a linear 
combination of J and A. (The Cayley-Hamilton theorem can be of help.) (b) Calculate ef4. 


1 O 1 0 0 1 —-1 0 
1A= : 2. A= : 3. A= : 4,A= ; 
1 1 b 2 1 O 0 1 


cost sin : 


—sint cost 


0 1 
5. (a) If A = ; , prove that et4 = 


b 


(b) Find a corresponding formula for e4 when A = , a, b real. 


—b a 
t—1 


; | prove that e¥) = eF(et-}), 


t 
6. If F(t) = 
o-[ 


7. If A(t) is a scalar function of ¢, the derivative of e4"!) is e4‘t)4’(t). Compute the derivative of 


t 
eAlt) when A(t) = Bef and show that the result is not equal to either of the two products 
eAlt) 4’(t) or A’ (te), 


In each of Exercises 8, 9, 10, (a) calculate A”, and express A? in terms of J, A, A*. (b) Calculate 
etA | 


01 1 01 1 2 0 0 
8 A=|0 0 1], 9 A=]0 1 1]. 10..A=]0 1 O7. 
0 0 0 0 0 0 01 1 
0 —!1 0 
11. 1f A =|] 1 0 1] , express e4 as a linear combination of I, A, A?. 
0 1 0 
0 1 0 x xy y? 
12. If A =|2 0 2], prove thate4 = |2xy x* + y* 2xy|,wherex =coshland y =sinh1. 
01 0 y xy x 


13. This example shows that the equation e4t? = e4e® is not always true for matrix exponentials. 


1 1 1 —1 
Compute each of the matrices e4e? , eBe4, eA*B when A = , | and B = b 7 , and 


note that the three results are distinct. 


7.13 Putzer’s method for calculating e'4 


The Cayley-Hamilton theorem shows that the nth power of any n x n matrix A can be 
expressed as a linear combination of the lower powers I, A, A?,..., A”. It follows that 
each of the higher powers A"*!, A”*®, ... , can also be expressed as a linear combination of 
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I, A, A®,..., A”. Therefore, in the infinite series defining e’4, each term t*A*/k! with 
k > nis a linear combination of t*I, "A, t"A?,...,t"A"-1. Hence we can expect that e’4 
should be expressible as a polynomial in A of the form 


n—1 


(7.27) i= a()A*, 


where the scalar coefficients g,(t) depend on ¢t. Putzer developed two useful methods for 
expressing e’4 as a polynomial in A. The next theorem describes the simpler of the two 
methods. 


THEOREM 7.9. Let A,,...,4, be the eigenvalues of an n X n matrix A, and define a 
sequence of polynomials in A as follows: 


(7.28)  P(AV=I, P(AV=][(A—Apl), for k=1,2,...50. 


Then we have 


n—1 
(7.29) ef == 22 n+i(t)P,(A), 
k= 
where the scalar coefficients r,(t),...,1,(t) are determined recursively from the system of 


linear differential equations 


r3(t) = Ayr,(t), r,(O) = 1, 


7.30 
Teyit) = ApsiTera(t) + 72 (2), rp41(0) = 0, (K=1, 2.242, —1). 
Note: Equation (7.29) does not express e'4 directly in powers of A as indicated in 

(7.27), but as a linear combination of the polynomials P(A), P,(A), ..., Pn_1(A). These 
polynomials are easily calculated once the eigenvalues of A are determined. Also the 
multipliers r,(¢), ... , r,(¢) in (7.30) are easily calculated. Although this requires solving 
a system of linear differential equations, this particular system has a triangular matrix 
and the solutions can be determined in succession. 


Proof. Let r,(t),...,7,(t) be the scalar functions determined by (7.30) and define a 
matrix function F by the equation 


(7.31) F(t) = > rea(OPA(A). 


Note that F(0) = 7,(0)P)(A) =I. We will prove that F(t) = e’4 by showing that F 
satisfies the same differential equation as e’4, namely, F’(t) = AF(t). 
Differentiating (7.31) and using the recursion formulas (7.30) we obtain 


F(t) = ¥ rhaOP A) = ¥ fn) as Aysalei(t)}P,(A), 
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where ro(t) is defined to be 0. We rewrite this in the form 


n—2 n—1 
FO) = ¥ rePrlA) + ¥ dete OPAA), 


then subtract A, F(t) = >) A,n41()P,(A) to obtain the relation 


(7.32) F() — AFD =S realO(PevslA) + Caer — ADP}. 
But from (7.28) we see that P,,,(A) = (A — AyuiD)P,(A), so 
Ppii(A) + nyt _ A.) Py,(A) = (A _ Anil) P(A) 5G Ait “1 h,,)P,(A) 


= (A —A,DP,(A). 


Therefore Equation (7.32) becomes 
n—2 
F'(t) ai A,F(t) 7, (A cn And) & PeOP(A) cas (A aa 4,1) F(t) = r(t)P»\(A)} 
The Cayley-Hamilton theorem implies that P,,(A) = O, so the last equation becomes 
Fi(t) — 4,F(t) = (4 — 4,DF@) = AF(t) — 4,F(), 


from which we find F’(t) = AF(t). Since F(O) = J, the uniqueness theorem (Theorem 7.7) 
shows that F(t) = et4. 


EXAMPLE |. Express e’4 as a linear combination of Jand A if Aisa2 xX 2 matrix with both 
its eigenvalues equal to A. 


Solution. Writing 4, = A, = A, we are to solve the system of differential equations 


ri(t) = Ar,(t), r,(0) = 1, 
rt) = Art) + rt), r(0) = 0. 
Solving these first-order equations in succession we find 
r(t)=e*, —r(t) = te**. 
Since P)(A) = J and P,(A) = A — AI, the required formula for e’+ is 


(7.33) eft = ef 4 te*(A — AT) = e*(1 — ADI + te*A. 
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EXAMPLE 2. Solve Example 1 if the eigenvalues of A are A and uw, where A ¥ wm. 


Solution. In this case the system of differential equations 1s 


ry(t) = Ar,(t), r,(0) = 1, 
ri) =pr() +r), — r,(0) =0. 


Its solutions are given by 


At __ elt 


r(t) = ent r(t) = 
A— 


Since P(A) = J and P,(A) = A — Al the required formula for e’4 is 


At __ pat ut, At At out 
(7.34) seg ee (A — al = he pen 4 & e 
Ae A— yu A—p 


If the eigenvalues A, w are complex numbers, the exponentials e** and e** will also be 
complex numbers. But if A and u are complex conjugates, the scalars multiplying J and A in 
(7.34) will be real. For example, suppose 


A=a+t ip, w=a— ip, pb #0. 
Then 4 — uw = 2if so Equation (7.34) becomes 


elatipit _ e(%—tB)t 


eA — oferty 4 —_____—_ [4 — (q + if)I] 
2ip 
za ipty re eb a ary : BD) 
= a = 4 
e f = a |! 


= o*{(cos Bt + isin Bt)I + _ (A —al — ip). 


The terms involving i cancel and we get 


(7.35) e'4 — 7 {(6 cos Bt — «sin Bt)I + sin Bt A}. 


7.14 Alternate methods for calculating e’“ in special cases 


Putzer’s method for expressing e’4 as a polynomial in A is completely general because it 
is valid for all square matrices A. A general method is not always the simplest method to 
use in certain special cases. In this section we give simpler methods for computing e‘4 in 
three special cases: (a) When all the eigenvalues of A are equal, (b) when all the eigenvalues 
of A are distinct, and (c) when A has two distinct eigenvalues, exactly one of which has 
multiplicity 1. 
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THEOREM 7.10. Jf A is ann X n matrix with all its eigenvalues equal to A, then we have 


n—1 
one SS Fe RI 
(7.36) et me DanG aD. 


Proof. Since the matrices At] and t(A — AI) commute we have 
Sak 

td _ atl t(A—al) __ (At Veg ane 

ee" = ee = (e D2 Al)”. 


The Cayley-Hamilton theorem implies that (A — AI)" = O for every k > n, so the theorem 
is proved. 


THEOREM 7.11. Jf A is ann X n matrix with n distinct eigenvalues i,, Ag, ...,4,, then 
we have 


eA = > e*L,(A) , 
k=1 
where L,(A) is a polynomial in A of degree n — 1 given by the formula 


n aw 
L,A) = |] oa for k=1,2,...,n. 
j=1 Ay. —~ M5 
j#k 
Note: The polynomials L;,(A) are called Lagrange interpolation coefficients. 


Proof. We define a matrix function F by the equation 


(7.37) F(t) = y e'*, (A) 


and verify that F satisfies the differential equation F’(t) = AF(t) and the initial condition 
F(0) = 7. From (7.37) we see that 


AF(t) — F(t) => e*(A — 4,DL,(A). 


By the Cayley-Hamilton theorem we have (A — A,I)L,(A) = O for each k, so F satisfies 
the differential equation F’(t) = AF(t). 

To complete the proof we need to show that F satisfies the initial condition F(0) = J, 
which becomes 


(7.38) > LA) = 1. 
k=1 
A proof of (7.38) is outlined in Exercise 16 of Section 7.15. 


The next theorem treats the case when A has two distinct eigenvalues, exactly one of 
which has multiplicity 1. 
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THEOREM 7.12. Let A be ann X n matrix (n > 3) with two distinct eigenvalues A and p, 
where A has multiplicity n — 1 and uw has multiplicity 1. Then we have 


, . n—2 tk (A aD} i ett et n—2 te 4 : A 4 ; 
= a ae ee > —(u — — Arr 
e e Lk! (u — a)” 1 (u — A) ae | ) 


Proof. As in the proof of Theorem 7.10 we begin by writing 


oo +k n—2 tk o) tk 
t4__ jt S k _ pat ban k At a k 
ee“ =e Did — at =e DA — +e > {Ar w 
k=0 k=0 k=n—1 
n—2 


t* 2) pr—ltr 
a A ee A pe et n—1+r 
= et DA ah tet (A — AT, 


m2 her) 


Now we evaluate the series over r in closed form by using the Cayley-Hamilton theorem. 
Since 
A—pl=A—AI—(u— A) 
we find 
(A —AD"-(A — wl) = (A — ADT)N" — (ue — AYA -— AD™. 


The left member is O by the Cayley-Hamilton theorem so 
(A — AD” = (uw — AA - AT)”. 
Using this relation repeatedly we find 
(A — ADT? = (wu — AA — AD. 


Therefore the series over r becomes 


— n—l+r 1 a . 
2 ae cae (u — a pS aC — AA — AT" 


=n—1 
= as t(u—A) oe = Aj (A = AI)" 
-Ga coi ial | 
This completes the proof. 


The explicit formula in Theorem 7.12 can also be deduced by applying Putzer’s method, 
but the details are more complicated. 

The explicit formulas in Theorems 7.10, 7.11 and 7.12 cover all matrices of ordern < 3. 
Since the 3 x 3 case often arises in practice, the formulas in this case are listed below for 
easy reference. 


CASE 1. If a3 x 3 matrix A has eigenvalues A, A, A, then 


eft = eT 4+ 1(A — Al) + HHA — AD}. 
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CASE 2. If a3 x 3 matrix A has distinct eigenvalues A, uw, v, then 


gtd — ptt (A - MDA) ou (A = AINA — v1) 4 ort (A — AIA = wD) 
(A — wy(A — ») (u — Au — ») (vy — A\(y — p) 


CASE 3. Ifa 3 x 3 matrix A has eigenvalues A, A, uw, with A ¥ pw, then 


ent = ett te** 
e'4 = eI + (A — AD} + ——[(A — Al’ — —- (A — AD’. 
(u — A) eA 
0 0 
EXAMPLE. Compute e“4 whenA=|0 OO 1 
2 —3) 4 


Solution. The eigenvalues of A are 1, 1, 2, so the formula of Case 3 gives us 
(7.39) 4 —e{1 + 1(A—DI} +(e —eyA —I — te(A— I)’. 
By collecting powers of A we can also write this as follows, 
(7.40) e'4 = (—2te + e**)I + ((3t + Qe — 284 — {(1t + le’ — eA’. 


At this stage we can calculate (A — J)? or A? and perform the indicated operations in (7.39) 
or (7.40) to write the result as a 3 x 3 matrix, 


—2tet + et (3t + 2)et— 2% —(t+ let + ec 
e4— | 20+ Det +2e% (GBr+ Set—4e%  —(t + et + 2c]. 
—2(t+2et+4e (3f+ 8)et— 8e%  —(t + 4)et + 4e2! 


7.15 Exercises 


For each of the matrices in Exercises 1 through 6, express e’4 as a polynomial in A. 


1 0 2 
a, 2 1 2 
1A= . j2aA= 3,.M4=]0 1 3 
4 -1 2 1 
0 0 1 
1 1 0 
0 1 oO 3 -1 1 
02 1 0 
4,.A=]| 0O 0 1 5.4=/]2 0 1 6. A = 
00 3 0 
—6 -ll —-6 L =f 2 
00 0 4 


7. (a) A3 x 3 matrix A is known to have all its eigenvalues equal to A. Prove that 
ed = belt{ (A222 — 2At + 2)1 + (—24r + 2NA 4+ 2A}. 


(b) Find a corresponding formula if A is a 4 x 4 matrix with all its eigenvalues equal to A. 
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In each of Exercises 8 through 15, solve the system Y'’=AY subject to the given initial 
condition. 
| 2. Cy —5 3 
8. A= , YO = . 9, A= ‘ Y(0) = 
2 -1 Co —15 7 
3 —-1 l 1 2 0 0 C 
10. A =|2 0 lj, YO) =} -1 ll. A=|]0 1 Of, Y(O) = | co 
1 —1 2 2 01 1 C3 
0 1 0 l —2 2 -3 8 
12. A= 0 0 1], YO) = | 0 13. A= 2 1 —-6|, Y(O) = | 0 
—6 -ll -—6 0 —j1 -—2 0 0 
1100 0 002 0 l 
02 1 0 1 00 2 0 
14.A = ; YO) = 1S. A= : Y(0) = ; 
0 0 3 0 00 0 4 2 
000 4 1 001 0 | 


16. This exercise outlines a proof of Equation (7.38) used in the proof of Theorem 7.11. Let Z,(A) 
be the polynomial in 4 of degree n — 1 defined by the equation 


Los TT 


j=1 
j#k 


where /4,,..., 4, are n distinct scalars. 
(a) Prove that 


0 


L,(4,) = 


(b) Let y,,... 


A Ae 


if A, ¥ A,, 


> Yn be n arbitrary scalars, and let 


pa) = > ypLy(A). 
k=1 


Prove that p(A) is the only polynomial of degree < n — 1 which satisfies the n equations 


Ps) = Ye 


for k =1,2,...,n. 


(c) Prove that ae L,(4) = 1 for every A, and deduce that for every square matrix A we have 


> 1,(A) =I, 
k=1 


where J is the identity matrix. 
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7.16 Nonhomogeneous linear systems with constant coefficients 


We consider next the nonhomogeneous initial-value problem 
(7.41) Y'(t)= AY(t)+ Q(0), Y(a) = B, 
on an interval J. Here Ais ann xX nconstant matrix, Q is an n-dimensional vector function 
(regarded as ann X 1 column matrix) continuous on J, and ais a given point inJ. We can 
obtain an explicit formula for the solution of this problem by the same process used to 
treat the scalar case. 

First we multiply both members of (7.41) by the exponential matrix e~‘-! and rewrite the 
differential equation in the form 


(7.42) e4fy"(t) — AY()} =e “QO(t). 


The left member of (7.42) is the derivative of the product e*-! Y(t). Therefore, if we 
integrate both members of (7.42) from a to x, where x € J, we obtain 


e *4Y(x) — e™4Y(a) = [° e'40(t) dt. 


Multiplying by e*4 we obtain the explicit formula (7.43) which appears in the following 
theorem. 


THEOREM 7.13. Let A be ann Xn constant matrix and let Q be an n-dimensional vector 
function continuous on an interval J. Then the initial-value problem 


Y(‘V=AYH)+ O00), YQ=B, 
has a unique solution on J given by the explicit formula 


x 
a 


(7.43) ¥(x) = 4B + ott [* o4Q(1) dt. 


As in the homogeneous case, the difficulty in applying this formula in practice lies in the 
calculation of the exponential matrices. 

Note that the first term, e'*-”)4B, is the solution of the homogeneous problem Y’(t) = 
AY(t), Y(a) = B. The second term is the solution of the nonhomogeneous problem 


Y()=AY(1)+ Q(t), Ya=O. 
We illustrate Theorem 7.13 with an example. 
EXAMPLE. Solve the initial-value problem 


Y()=AY(t)+ Q(t), YO)=B, 
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on the interval (— oo , +0), where 


2 —-1 1] et 
A=1|0 3 —l], O(t)=]| 0 |, B= 
2 l 3 ter! 


Solution. According to Theorem 7.13, the solution is given by 


(7.44) ¥(x) = 4 |” e4Q(n dt = |” eP4Q(H) at. 


The eigenvalues of A are 2, 2, and 4. To calculate e*4 we use the formula of Case 3, 
Section 7.14, to obtain 


e*4 = e {JT + x(A — 21)} + h(e** — e*\(A — 21) — 3xe?*(A — 21) 
= {1 + x(A — 21) + Ue — 2x — 1A — 20%}. 


We can replace x by x — ¢ in this formula to obtain e’*-’4, Therefore the integrand in 
(7.44) is 


el P40 (1) = MOT + (x — A — 21) + ae? — 2A(x — 1) — 1A — 213000) 


1 x—t ete _ ax — th —1 
= e*!0| +(A — 2I)e* 0 4 AG — 21)*e** 0 
t t(x — t) e**te"** — 2t(x —t)—t 
Integrating, we find 
x oe 
Y(x) -| e'*)40(t) dt = e*| 0 | + (A — 2De™*| 0 
0 
1y? 1,3 
2 
; be’? —$—x — x? 
+=(A — 21)%e* 0 
4 de® —1—1y —],? — 1,3 
Since we have 
0 -1 | 2 0 2 
A—2I1=1|0 1 -l and (A—21?=]-2 O -2!], 
2 1] ] 2 0 2 
we find 
x ax jet xb 
Y(x) =e"! 0 | +e) —ax? | + | Fe" + § + dx t+ be + Ho 
aXe Sal ab. ge — 8 — 3x — 9x? — Gx8 


ie ato — be 
= e7*| —2e% 4 2 + Bx + Bx? 1, 
ie" Fie t be 


The rows of this matrix are the required functions y,, yo, ys. 
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7.17. Exercises 


1. Let Z be a solution of the nonhomogeneous system 
Z(t) = AZ(t) + Q(t), 


on an interval J with initial value Z(a). Prove that there is only one solution of the non- 
homogeneous system 


Y'(t) = AY(t) + Q(t) 
on J with initial value Y(a) and that it is given by the formula 
Y(t) = Z(t) + et M4{ Ya) — Za}. 


Special methods are often available for determining a particular solution Z(t) which resembles 
the given function Q(¢). Exercises 2, 3, 5, and 7 indicate such methods for Q(t) = C, Q(t) = 
eC, O(t) = tC, and Q(t) = (cos at)C + (sin at)D, where C and D are constant vectors. 
If the particular solution Z(t) so obtained does not have the required initial value, we modify 
Z(t) as indicated in Exercise 1 to obtain another solution Y(t) with the required initial 
value. 

2. (a) Let A be aconstant m x n matrix, B and C constant n-dimensional vectors. Prove that the 
solution of the system 


Y(t) =AY() + C, Y(a) = B, 


on (— ©, +) is given by the formula 
Y(x) = eft 4B + (|* “ena du)C. 


(b) If A is nonsingular, show that the integral in part (a) has the value {e‘*-#4 — J}A7, 
(c) Compute Y(x) explicitly when 


Lah ech e-[f me 


3. Let A be ann xX nconstant matrix, let B and C be n-dimensional constant vectors, and let « 
be a given scalar. 
(a) Prove that the nonhomogeneous system Z ‘(t) = AZ(t) + eC has a solution of the form 
Z(t) = eB if, and only if, («J — AJB =C. 
(b) If « is not an eigenvalue of A, prove that the vector B can always be chosen so that the 
system in (a) has a solution of the form Z(t) = e*B. 
(c) If « is not an eigenvalue of A, prove that every solution of the system Y(t) = A Y(t) + e#C 
has the form Y(t) = e4( Y(O) — B) + eB, where B = (aI — A)!C. 

4. Use the method suggested by Exercise 3 to find a solution of the nonhomogeneous system 
Y(t) =AY() + eC, with 


hd ef} =f 
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5. Let A be ann x n constant matrix, let B and C be n-dimensional constant vectors, and let 
m be a positive integer. 
(a) Prove that the nonhomogeneous system Y(t) = AY(t) + t"C, YO) = B,hasa particular 
solution of the form 


Y() = By + (By + CB, + +++ +t"B,, 


where By, B,,..., B» are constant vectors, if and only if 
1 
C= —-—AmrlB, 
m! 


Determine the coefficients B,, B,,..., B,, for such a solution. 
(b) If A is nonsingular, prove that the initial vector B can always be chosen so that the system 
in (a) has a solution of the specified form. 

6. Consider the nonhomogeneous system 


yi =3y +yet+h 


y, =2y, +27. + 8. 


(a) Find a particular solution of the form Y(t) = By + 4B, + By, + ABs. 
(b) Find a solution of the system with y,(0) = y,(0) = 1. 

7. Let A be ann x n constant matrix, let B, C, D be n-dimensional constant vectors, and let 
a be a given nonzero real number. Prove that the nonhomogeneous system 


Y(t) = AY(t) + (cos at)C + (sin at)D, Y(O) = B, 
has a particular solution of the form 
Y(t) = (cos «f)E + (sin «t)F, 
where E and F are constant vectors, if and only if 
(A? + @IDB = —(AC + «D). 


Determine E and Fin terms of A, B, Cfor sucha solution. Note that if A? + «77 is nonsingular, 
the initial vector B can always be chosen so that the system has a solution of the specified 
form. 


8. (a) Find a particular solution of the nonhomogeneous system 
Yi =) + By. + 4 5in 2¢ 
Yo =i — Yas 


(b) Find a solution of the system with y,(0) = y,(0) = 1. 
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In each of Exercises 9 through 12, solve the nonhomogeneous system Y'(t) = AY(t) + O(1) 
subject to the given initial condition. 


= ee | 0 rid 6 
12,A=|] 0 -1 —1], Q(t) =] 2r|, Y(O) = | -2]. 
0 oO -1 t 1 


7.18 The general linear system Y’(t) = P(t) Y(t) + Q(t) 
Theorem 7.13 gives an explicit formula for the solution of the linear system 
Y(t)=AY)+ Qt), Ya=B, 


where A is a constant m X n matrix and Q(t), Y(t) are n x 1 column matrices. We turn 
now to the more general case 


(7.45) Y'(t) = P(t) Y(t) + O(1), Y(a) = B, 


where the n X n matrix P(t) is not necessarily constant. 

It P and Q are continuous on an open interval J, a general existence-uniqueness theorem 
which we shall prove in a later section tells us that for each a in J and each initial vector B 
there is exactly one solution to the initial-value problem (7.45). In this section we use this 
result to obtain a formula for the solution, generalizing Theorem 7.13. 

In the scalar case (n = 1) the differential equation (7.45) can be solved as follows. We 
let A(x) = J P(t) dt, then multiply both members of (7.45) by e~4™ to rewrite the differ- 
ential equation in the form 


(7.46) e 4%) — P()Y(t)} = ec 4%O(t). 


Now the left member is the derivative of the product e-4 Y(t). Therefore, we can integrate 
both members from a to x, where a and x are points in J, to obtain 


ey (x) as e 4 y(q) a= [? e AMO(1) dt. 
Multiplying by e4 we obtain the explicit formula 


(7.47) ¥(x) = e@eAy (a) 4 e4'® |* oA QL) dt 
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The only part of this argument that does not apply immediately to matrix functions is 
the statement that the left-hand member of (7.46) is the derivative of the product e-4™ Y(t). 
At this stage we used the fact that the derivative of e-4™ is —P(t)e-4™. In the scalar case 
this is a consequence of the following formula for differentiating exponential functions: 


If E(t) = e4 then E(t) = A’(te4™. 
Unfortunately, this differentiation formula is not always true when A is a matrix function. 
1 ¢ 
For example, it is false for the 2 x 2 matrix function A(t) = ; . (See Exercise 7 of 
0 


Section 7.12.) Therefore a modified argument is needed to extend Equation (7.47) to the 
matrix case. 


Suppose we multiply each member of (7.45) by an unspecified n x n matrix F(t). This 
gives us the relation 


F(t) Y(t) = F()P(t) Y(t) + FQ). 


Now we add F(t) Y(t) to both members in order to transform the left member to the 
derivative of the product F(t) Y(t). If we do this, the last equation gives us 


{F(t) Y(t)}’ = {F'(t) + F)P()} Y(t) + F(NQ(t). 


If we can choose the matrix F(t) so that the sum {F’(t) + F(t)P(t)} on the right is the zero 
matrix, the last equation simplifies to 


(FQ) Y(t} = FOC). 


Integrating this from a to x we obtain 
F(x) ¥(x) — F(a)Y(a) = | : F(t)Q(t) dt. 


If, in addition, the matrix F(x) is nonsingular, we obtain the explicit formula 


(7.48) ¥(x) = F(x) 7F(a)¥(a) + F(x |* FOU at. 


This is a generalization of the scalar formula (7.47). The process will work if we can find a 
n X n matrix function F(t) which satisfies the matrix differential equation 


F'(t) = —F(t)P(t) 
and which is nonsingular. 

Note that this differential equation is very much like the original differential equation 
(7.45) with Q(t) = 0, except that the unknown function F(t) 1s a square matrix instead of a 
column matrix. Also, the unknown function is multiplied on the right by —P(¢) instead of 
on the left by P(t). 

We shall prove next that the differential equation for F always has a nonsingular solution. 
The proof will depend on the following existence theorem for homogeneous linear systems. 
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THEOREM 7.14. Assume A(t) is ann X n matrix function continuous on an open interval 
J. Ifa éJ and if B is a given n-dimensional vector, the homogeneous linear system 


Y'(t) = A(t) Y(t), Y(a) = B, 


has an n-dimensional vector solution Y on J. 


A proof of Theorem 7.14 appears in Section 7.21. With the help of this theorem we can 
prove the following. 


THEOREM 7.15. Given an n Xn matrix function P, continuous on an open interval J, 
and given any point ain J, there exists ann X n matrix function F which satisfies the matrix 
differential equation 
(7.49) F'(x) = —F(x)P(x) 
on J with initial value F(a) = I. Moreover, F(x) is nonsingular for each x in J. 

Proof. Let Y,(x) be a vector solution of the differential equation 

V(x) = —P(x)'¥,(x) 
onJ with initial vector Y,(a) = J,,, where J, is the kth column of them x nidentity matrix J. 
Here P(x)! denotes the transpose of P(x). Let G(x) be then x n matrix whose kth column 
is Y,(x). Then G satisfies the matrix differential equation 


(7.50) G'(x) = —P(x)'G(x) 


on J with initial value G(a) = J. Now take the transpose of each member of (7.50). Since 
the transpose of a product is the product of transposes in reverse order, we obtain 


{G'(x)}* = —G(x)'P(x). 

Also, the transpose of the derivative G’ is the derivative of the transpose G’. Therefore the 
matrix F(x) = G(x)! satisfies the differential equation (7.49) with initial value F(a) = /. 

Now we prove that F(x) is nonsingular by exhibiting its inverse. Let H be then x n 
matrix function whose kth column is the solution of the differential equation 

Y'(x) = P(x) Y(x) 
with initial vector Y(a) = J,, the kAthcolumn of J. Then H satisfies the initial-value problem 
H'(x) = P(x)H(x), Ha) =I, 


on J. The product F(x)H(x) has derivative 


F(x)H'(x) + F’'(x)H(x) = F(x) P(x)H(x) — F(x) P(x) H(x) = O 
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for each x in J. Therefore the product F(x)H(x) is constant, F(x)H(x) = F(a)H(a) = T, 
so H(x) is the inverse of F(x). This completes the proof. 


The results of this section are summarized in the following theorem. 


THEOREM 7.16. Given ann X n matrix function P and an n-dimensional vector function 
Q, both continuous on an open interval J, the solution of the initial-value problem 


(7.51) Y'(x) = P(xX)Y¥x)+ O00), YaQ=B, 
on J is given by the formula 


(7.52) ¥(x) = F(x)¥(a) + Foxy? |* F(DQ(D at. 


Then X n matrix F(x) is the transpose of the matrix whose kth column is the solution of the 
initial-value problem 


(7.53) Y'(x) = —P(x)'¥(x), Ya@=I,, 


where I, is the kth column of the identity matrix I. 


Although Theorem 7.16 provides an explicit formula for the solution of the general 
linear system (7.51), the formula is not always a useful one for calculating the solution 
because of the difficulty involved in determining the matrix function F. The determination 
of F requires the solution of n homogeneous linear systems (7.53). The next section 
describes a power-series method that is sometimes used to solve homogeneous linear 
Systems. 

We remind the reader once more that the proof of Theorem 7.16 was based on Theorem 
7.14, the existence theorem for homogeneous linear systems, which we have not yet proved. 


7.19 A power-series method for solving homogeneous linear systems 


Consider a homogeneous linear system 
(7.54) Y"(x) = A(x) Y(x), Y(0) = B, 


in which the given n X n matrix A(x) has a power-series expansion in x convergent in 
some open interval containing the origin, say 


Al) = Aye ap se A ee ee Age iets for |x| <r,, 


where the coefficients Aj, A,;, A2,... are given n Xn matrices. Let us try to find a 
power-series solution of the form 


Y(x) = By + xB, + x*B, +e:+ + x°B, + ¢°°, 
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with vector coefficients B,, B,, B,,.... Since Y(0) = By, the initial condition will be 
satisfied by taking By = B, the prescribed initial vector. To determine the remaining 
coefficients we substitute the power series for Y(x) in the differential equation and equate 
coefficients of like powers of x to obtain the following system of equations: 


k 
(7.55) B, = AoB , (k + 1) Bust =) A,B,_, for k = f. 2. elie te 
r=0 
These equations can be solved in succession for the vectors B,, B,,.... If the resulting 


power series for Y(x) converges in some interval |x| < r,., then Y(x) will be a solution of the 
initial-value problem (7.54) in the interval |x| <r, where r = min {r,, ro}. 

For example, if A(x) is a constant matrix A, then Ay = A and A, = O fork > 1, so the 
system of equations in (7.55) becomes 


Solving these equations in succession we find 


By =~ AB for k>1. 


Therefore the series solution in this case becomes 
Y(x) = B+ >= A®B = e*4B. 
ame k! 


This agrees with the result obtained earlier for homogeneous linear systems with constant 
coefficients. 


7.20 Exercises 


1. Let p be a real-valued function and Q ann x 1 matrix function, both continuous on an interval 
J. Let A be ann x nconstant matrix. Prove that the initial-value problem 


Y'(x) = p(x)AY(x) + Q(x), Y(a) = B, 
has the solution 
Y(x) = ear) AB + ed et) AQ(?r) dt 


on J, where g(x) = Jf p(t) dt. 
2. Consider the special case of Exercise 1 in which A is nonsingular, a = 0, p(x) = 2x, and 
Q(x) = xC, where C is a constant vector. Show that the solution becomes 


Y(x) = e”4(B +447C) — 44°C. 


3. Let A(t) be an n x n matrix function and let E(t) = e4™. Let Q(t), Y(t), and B ben x 1 
column matrices. Assume that 


E'(t) = A’ (E(t) 
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on an open interval J. If a€ J and if A’ and Q are continuous on J, prove that the initial-value 
problem 
YO=AOYO+O0, Y@= 


has the following solution on J: 


Y(x) = e4(@)e-Ala) B + a) he e-Al)O(t) dt. 


4, Let E(t) = e4™. This exercise describes examples of matrix functions A(t) for which E’(t) = 
A'(t)E(t). 
(a) Let A(t) = t"A, where A is ann x nconstant matrix andrisa positive integer. Prove that 
E'(t) = A’(t)E(t) on (— ~, ©). 
(b) Let A(t) bea polynomial i in ¢ with matrix coefficients, say 


m 
A(t) = > t"A,, 
r=0 
where the coefficients commute, A,A, = A,A, for all r and s. Prove that E’(t) = A’(r)E(t) on 


( —, oO), 
(c) Solve the homogeneous linear system 


Y(t) =(1 + tA) Y(O), Y(0) = 
on the interval (— 0, 0), where A is ann x nconstant matrix. 
5. Assume that the n x n matrix function A(x) has a power-serieés expansion convergent for 
|x| <r. Develop a power-series procedure for solving the following homogeneous linear system 
of second order: 


Y’(x) = A(x) Y(x), with Y(0) = B, Y'(0) = 


6. Consider the second-order system Y’(x) + AY(x) =0, with Y(O) = B, Y’(0) = C, where A 
is a constant n x n matrix. Prove that the system has the power-series solution 


a (—1)*x?* 4% (—1)Fx2ht1 4k 
ae (i >a (2k)! )p +(x v Sarsar Qk+yr }° 
convergent for —o <x < +0. 


7.21 Proof of the existence theorem by the method of successive approximations 
In this section we prove the existence and uniqueness of a solution for any homogeneous 
linear system 


(7.56) Y(t) = A(t) ¥(1), 


where A(t) is ann X n matrix function, continuous on an open interval J. We shall prove 
that for any point a in J and any given initial-vector B there exists exactly one solution Y(t) 
on J satisfying the initial condition Y(a) = 
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We shall use the method of successive approximations, an iterative method which also has 
applications in many other problems. The method was first published by Liouville in 1838 
in connection with the study of linear differential equations of second order. It was later 
extended by J. Caqué in 1864, L. Fuchs in 1870, and G. Peano in 1888 to the study of linear 
equations of order n. In 1890 Emile Picard (1856-1941) extended the method to encompass 
nonlinear differential equations as well. In recognition of his fundamental contributions, 
some writers refer to the method as Picard’s method. The method is not only of theoretical 
interest but can also be used to obtain numerical approximations to solutions in some cases. 

The method begins with an initial guess at a solution of the equation (7.56). We take as 
initial guess the given initial vector B, although this is not essential. We then substitute 
this guess in the right-hand member of the equation and obtain a new differential equation, 


Y(t) = A(t)B. 
In this equation the right-hand member no longer contains the unknown function, so the 
equation can be solved immediately by integrating both members from a to x, where x is 


an arbitrary point in J. This equation has exactly one solution Y, on J satisfying the initial 
condition Y,(a) = B, namely 


¥(x) = B+ |* AWB at. 


Now we replace Y(t) by Y,(t) in the right-hand member of the original differential 
equation (7.56) to obtain a new differential equation 


Y'(t) = A(t) Y,(t). 
This equation has a unique solution Y, on J with Y,(a) = B, 
(7.57) V(x) = B+ |” AYO dt. 


We then substitute Y, in the right-hand member of (7.56) and solve the resulting equation 
to determine Y; with Y;(a) = B, and so on. This process generates a sequence of functions 


Yo, Y1, Y2,..., where Y) = B and where Y,,, is determined from Y,, by the recursion 
formula 
(7.58) Yu) = B+] AY ()dt for k=0,1,2,.... 


Our goal is to prove that the sequence of functions so defined converges to a limit 
function Y which is a solution of the differential equation (7.56) on J and which also satisfies 
the initial condition Y(a) = B. The functions Yo, Y,, Yo,...arecalled successive approxi- 
mations to Y. Before we investigate the convergence of the process we illustrate the method 
with an example. 


EXAMPLE. Consider the initial-value problem Y’(t) = AY(t), Y(O) = B, where A is a 
constant n X n matrix. We know that the solution is given by the formula Y(x) = e*4B 
for all real x. We will show how this solution can be obtained by the method of successive 
approximations. 
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The initial guess is Yo(x) = B. The recursion formula (7.58) gives us 


¥(x) = B+ |” AB dt = B+ xAB, 


¥,(x) = B+ |” AY) dt = B+ |* (AB + tA°B) dt = B+ xAB + BPA’. 
By induction we find 


k r 
Y,(x) = B+ xAB + $x°A°B $+ +> xtAtB = (So), 
k! =a 0 


The sum on the right is a partial sum of the series for e*-'. Therefore when k —> 00 we find 


lim Y,(x) = e*4B 


k-> 0 


for all x. Thus, in this example we can show directly that the successive approximations 
converge to a solution of the initial-value problem on (— oo, +00). 


Proof of convergence of the sequence of successive approximations. We return now to the 
general sequence defined by the recursion formula (7.58). To prove that the sequence 
converges we write each term Y,(x) as a telescoping sum, 


(7.59) Ya(e) = Yo(x) + (Ynss(®) — Yon(2)}- 


To prove that Y,,(x) tends to a limit as k —> 0 we shall prove that the infinite series 


(7.60) > Yngi(X) — Yn(X)} 


eonverges for each x in J. For this purpose it suffices to prove that the series 


(7.61) Y WYaeis(2) — Yn 


converges. In this series we use the matrix norm introduced in Section 7.3; the norm of a 
matrix is the sum of the absolute values of all its entries. 

Consider a closed and bounded subinterval J, of J containing a. We shall prove that for 
every x in J, the series in (7.61) is dominated by a convergent series of constants inde- 
pendent of x. This implies that the series converges uniformly on J,. 

To estimate the size of the terms in (7.61) we use the recursion formula repeatedly. 
Initially, we have 


¥4(x) — Yo(x) = [* AB at. 
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For simplicity, we assume that a < x. Then we can write 


(7.62) IX) — Yoh = |] [7 ace aell < [* bac Bi ae. 


Since each entry of A(t) is continuous on J, each entry is bounded on the closed bounded 
interval J,. Therefore ||A(t)|| < 4, where ™ is the sum of the bounds of all the entries of 
A(t) on the interval J,. (The number M depends on J,.) Therefore the integrand in (7.62) 
is bounded by ||Bl| /, so we have 


IY) — Yo(xDI < J” PBIM dt = |B] Mx — a) 


for all x > ainJ,. 
Now we use the recursion formula once more to express the difference Y, — Y, in terms 
of Y, — Yo, and then use the estimate just obtained for Y, — Y, to obtain 


Ix) — YO =|] [* amen — yco} ae] < [* 1A@H BI MC a) at 


a M*(x — a)’ 
< |B M? |" (t — a) dt = Bj ME 
a 2! 
for all x > ainJ,. By induction we find 
M™! x—a m+1 
[Ynaxl2) — Ym < 1B) “A= 2 for m=0,1,2,..., 


(m + 1)! 


and for allx > ainJ,. Ifx < aasimilar argument gives the same inequality with |x — a| 
appearing instead of (x — a). If we denote by L the length of the interval J, , then we have 
|x — a| < L for all x in J, so we obtain the estimate 


2 Mmtipmtl F 0 , 5 
Ynewlx) — Yn(X)|l < || Bi) ——— or m=0,1,2,..., 
Il Yin t.1(%) (x) < Bll “apy 


and for all x inJ,. Therefore the series in (7.61) is dominated by the convergent series 


2 (ML)™*+* 


ere Neal Gian 


Bi 


This proves that the series in (7.61) converges uniformly on J,. 

The foregoing argument shows that the sequence of successive approximations alwavs 
converges and the convergence is uniform onJ,. Let Y denote the limit function. That is, 
define Y(x) for each x in J, by the equation 


Y(x) = lim Y,(x). 
k7 0 
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We shall prove that Y has the following properties: 

(a) Y is continuous on J,. 

(b) Y(x)= B+ feA@MY(t)dt forall x inJ,. 

(c) Y(a)=B and Y'(x) = A(x) Y(x) for all x in J,. 
Part (c) shows that Y is a solution of the initial-value problem on J,. 


Proof of (a). Each function Y, is a column matrix whose entries are scalar functions, 
continuous on J,. Each entry of the limit function Y is the limit of a uniformly convergent 
sequence of continuous functions so, by Theorem 11.1 of Volume I, each entry of Y is also 
continuous on J,. Therefore Y itself is continuous on J,. 


Proof of (b). The recursion formula (7.58) states that 


Yen) = B+ |* AMY dt. 


Therefore 
¥(x) = lim Y,(x) = B +lim [* AWY,() dt = B+ [* A(Olim Y,(0) dt 
ko k7 0 *4 if kc 
=B+ [’ A(t) Y(t) dt. 


The interchange of the limit symbol with the integral sign is valid because of the uniform 
convergence of the sequence {Y,} on J,. 


Proof of (c). The equation Y(a) = B follows at once from (b). Because of (a), the 
integrand in (b) is continuous on J, so, by the first fundamental theorem of calculus, Y'(x) 
exists and equals A(x) Y(x) on J,. 

The interval J; was any closed and bounded subinterval of / containing a. If J, is 
enlarged, the process for obtaining Y(x) doesn’t change because it only involves integration 
from ato x. Since for every x in J there is a closed bounded subinterval of J containing a 
and x, a solution exists over the full interval J. 


THEOREM 7.17. UNIQUENESS THEOREM FOR HOMOGENEOUS LINEAR SYSTEMS. If A(t) is 
continuous on an open interval J, the differential equation 


Y'(t) = A(t) Y(t) 
has at most one solution on J satisfying a given initial condition Y(a) = B. 
Proof. Let Yand Z be two solutions on J. Let J; be any closed and bounded subinterval 
of J containing a. We will prove that Z(x) = Y(x) for every x in J,. This implies that 


Z = Y on the full interval J. 
Since both Y and Z are solutions we have 


Z'(t) — Y(t) = AQYZD — YO}. 
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Choose x in J, and integrate this equation from a to x to obtain 


Z(x) — ¥(x) = |” A(N{Z() — YO} dt. 


This implies the inequality 


(7.63) Zee) — Yoo < Mf? iz) — YCol ae 


>] 


where &M is an upper bound for || A(t)|| on J,. Let M4, be an upper bound for the continuous 
function ||Z(t) — Y(t)|| on J,. Then the inequality (7.63) gives us 


(7.64) |Z(x) — Y(x)l < MM, |x — al. 
Using (7.64) in the right-hand member of (7.63) we obtain 


Ix — al? 


|Z) — YO) < M’M, | [* jt — al at| = MOM, 


By induction we find 


Ix — al” 


(7.65) |Z(x) — Y(x)|| < M"M, 
m! 


When m —> oo the right-hand member approaches 0, so Z(x) = Y(x). This completes the 
proof. 


The results of this section can be summarized in the following existence-uniqueness 
theorem. 


THEOREM 7.18. Let A be an n Xn matrix function continuous on an open interval J. 
Ifa eJ and if B is any n-dimensional vector, the homogeneous linear system 


Y()V=AOYO), Y@Q=B, 
has one and only one n-dimensional vector solution on J. 


7.22 The method of successive approximations applied to first-order nonlinear systems 


The method of successive approximations can also be applied to some nonlinear systems. 
Consider a first-order system of the form 


(7.66) Y’ = F(t, Y), 


where Fis a given n-dimensional vector-valued function, and Yis an unknown n-dimensional 
vector-valued function to be determined. We seek a solution Y which satisfies the equation 


Y'(t) = Flt, Y(0)] 
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for each ¢ in some interval J and which also satisfies a given initial condition, say Y(a) = B, 
where a €J and B is a given n-dimensional vector. 

In a manner parallel to the linear case, we construct a sequence of successive approxi- 
mations Yj, Y,, Yo,..., by taking Y) = B and defining Y,,, in terms of Y, by the 
recursion formula 


(7.67) Yu =B+{" Fl, Y(Oldt for k=0,1,2,.... 


Under certain conditions on F, this sequence will converge to a limit function Y which will 
satisfy the given differential equation and the given initial condition. 

Before we investigate the convergence of the process we discuss some one-dimensional 
examples chosen to illustrate some of the difficulties that can arise in practice. 


EXAMPLE 1. Consider the nonlinear initial-value problem y’ = x? + y? with y = 0 when 
x = 0. We shall compute a few approximations to the solution. We choose Y,(x) = 0 
and determine the next three approximations as follows: 


40) = [ar = x 

x 6 3 7 
Y(x)= | [2 + Y(t a= ["(e er} = 
(x) [te + v0) “(e+ ) aat4% 


rae ep  ¢ 2x1! x 
Y,(x) = f+ (F+ a) 545 + + 
a(*) |, 3° 63 2079 59535 


15 


It is now apparent that a great deal of labor will be needed to compute further approxi- 
mations. For example, the next two approximations Y, and Y; will be polynomials of 
degrees 31 and 63, respectively. 

The next example exhibits a further difficulty that can arise in the computation of the 
successive approximations. 


EXAMPLE 2. Consider the nonlinear initial-value problem y’ = 2x + e”, with y = 0 when 
= 0. We begin with the initial guess Y,(x) = 0 and we find 


Y,(x) = {’ (20+ 1)dt=x*4+x, 


Y(x)= | (2Qttet)dt=xe+ fet tat. 
0 0 


Here further progress is impeded by the fact that the last integral cannot be evaluated in 
terms of elementary functions. However, for a given x it is possible to calculate a numerical 
approximation to the integral and thereby obtain an approximation to Y,(x). 

Because of the difficulties displayed in the last two examples, the method of successive 
approximations is sometimes not very useful for the explicit determination of solutions 
in practice. The real value of the method is its use in establishing existence theorems. 
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7.23 Proof of an existence-uniqueness theorem for first-order nonlinear systems 


We turn now to an existence-uniqueness theorem for first-order nonlinear systems. By 
placing suitable restrictions on the function which appears in the right-hand member of the 
differential equation 

YS FY). 


we can extend the method of proof used for the linear case in Section 7.21. 
Let J denote the open interval over which we seek a solution. Assume a € Jand let B bea 
given n-dimensional vector. Let S denote a set in (n + 1)-space given by 


S = {(x,Y) | Ix—al sh, ||Y— Bl <k}, 


whereh > Oandk > 0. [If = 1 this is a rectangle with center at (a, B) and with base 2h 
and altitude 2k.] We assume that the domain of F includes a set S of this type and that F is 
bounded on S, say 


(7.68) F(x, YI << M 


for all (x, Y)in S, where M is a positive constant. 

Next, we assume that the composite function G(x) = F(x, Y(x)) is continuous on the 
interval (a —h,a-+ h) for every function Y which is continuous on (a —h,a-+ A) and 
which has the property that (x, Y(x))€S for all x in (@a—h,a+h). This assumption 
guarantees the existence of the integrals that occur in the method of successive approxi- 
mations, and it also implies continuity of the functions so constructed. 

Finally, we assume that F satisfies a condition of the form 


|F(x, Y) — F(x, Z)|| < AY — Z| 


for every pair of points (x, Y) and (x, Z) in S, where A is a positive constant. This is called 
a Lipschitz condition in honor of Rudolph Lipschitz who first introduced it in 1876. A 
Lipschitz condition does not restrict a function very seriously and it enables us to extend the 
proof of existence and uniqueness from the linear to the nonlinear case. 


THEOREM 7.19. EXISTENCE AND UNIQUENESS OF SOLUTIONS TO FIRST-ORDER NONLINEAR 
SYSTEMS. Assume F satisfies the boundedness, continuity, and Lipschitz conditions specified 
above ona set S. Let I denote the open interval (a — c,a-+c), where c = min {h, k/M}. 
Then there is one and only one function Y defined on I with Y(a) = B such that (x, Y(x))ES 
and 

Y'(x) = F(x, Y(x)) ~— for each x in I. 


Proof. Since the proof is analogous to that for the linear case we sketch only the 
principal steps. We let Y)(x) = B and define vector-valued functions Y,, Y2,...on J by 
the recursion formula 


(7.69) Yau) =B+ )° Fl, Y,@]dt for m=0,1,2,.... 
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For the recursion formula to be meaningful we need to know that (x, Y,,,(x)) € S for each 
xin J. This is easily proved by induction onm. When m = 0 we have (x, Y(x)) = (x, B), 
which is in S. Assume then that (x, Y,,(x)) € S for some m and each x in J. Using (7.69) 
and (7.68) we obtain 


IY) — BI < | J” wFte Yc at] <M | [* at| = Mix — al. 


Since |x — a| <c for x in J, this implies that 
| Ynpi(x) — Bll < Me <k, 


which shows that (x, Y,,,;(x)) ¢ S for each x in J. Therefore the recursion formula is 
meaningful for every m > 0 and every x in J. 

The convergence of the sequence {Y,,,(x)} is now established exactly as in Section 7.21. 
We write 


Vx) = YX) + Y (Ynesl) — YC} 


and prove that Y,(x) tends to a limit as k —> o0 by proving that the infinite series 


¥ lYmulx) — Yao 


m=0 


converges on J. This is deduced from the inequality 


MA™ |x — al MA™c™1 


| Yinzi(X) = F(X) Ss (m + 1)! = (m + 1)! 


which is proved by induction, using the recursion formula and the Lipschitz condition. 
We then define the limit function Y by the equation 


Y(x) = hm Y,,(x) 


mo 


for each x in J and verify that it satisfies the integral equation 


¥(x)=B+ |* Flt, Yl at, 


exactly as in the linear case. This proves the existence of a solution. The uniqueness may 
then be proved by the same method used to prove Theorem 7.17. 


7.24 Exercises 


1. Consider the linear initial-value problem 


y +y =2e", with y=1whenx =0. 


3. 


4. 
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(a) Find the exact solution Y of this problem. 
(b) Apply the method of successive approximations, starting with the initial guess Yo(x) = 1. 
Determine Y,,(x) explicitly and show that 


lim Y,(x) = Y(x) 


N+ 


for all real x. 


. Apply the method of successive approximations to the nonlinear initial-value problem 


y=uxty’, with y =0 when x =0. 


Take Yo(x) = 0 as the initial guess and compute Y3(x). 
Apply the method of successive approximations to the nonlinear initial-value problem 


y=l+xy*, with y =0Owhen x =0. 


Take Y)(x) = 0 as the initial guess and compute Y3(x). 
Apply the method of successive approximations to the nonlinear initial-value problem 


y=exsty’, with y =0Owhen x =0. 


Start with the “bad” initial guess Yj(x) = 1, compute Y3(x), and compare with the results of 
Example 1 in Section 7.22. 


. Consider the nonlinear initial-value problem 


y =x + y?, with y=1whenx =0. 


(a) Apply the method of successive approximations, starting with the initial guess Y,(x) = 1, 
and compute Y,(x). 

(b) Let R = [—1,1] x [—1, 1]. Find the smallest M such that |f(x, y)| < Mon R. Findan 
interval J = (—c,, c) such that the graph of every approximating function Y,, over J will liein R. 
(c) Assume the solution y = Y(x) has a power-series expansion in a neighborhood of the 
origin. Determine the first six nonzero terms of this expansion and compare with the result of 


part (a). 


. Consider the initial-value problem 


y=lt+y’, with y =0Owhen x =0. 


(a) Apply the method of successive approximations, starting with the initial guess Y)(x) = 0, 
and compute Y,(~). 

(b) Prove that every approximating function Y,, is defined on the entire real axis. 

(c) Use Theorem 7.19 to show that the initial-value problem has at most one solution in any 
interval of the form (—/, hf). 

(d) Solve the differential equation by separation of variables and thereby show that there is 
exactly one solution Y of the initial-value problem on the interval (— 7/2 , 7/2) and no solution 
on any larger interval. In this example, the successive approximations are defined on the entire 
real axis, but they converge to a limit function only on the interval (— 7/2 , 7/2). 


. We seek two functions y = Y(x) and z = Z(x) that simultaneously satisfy the system of 


equations 
yorz, 2 =x(y +2) 
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with initial conditions y = 1 andz = 1/2 when x = 0. Start with the initial guesses Yo(x) = 1, 
Z (x) = 1/2, and use the method of successive approximations to obtain the approximating 
functions 


3 3) x6 x? 


XxX xX 
¥0) =1+5+ 9) += + fo2> 
Z.(x) 1 3x* x 3x8 7x9 x 
8) => +3 +19 + Gq + 360 tT 256° 


8. Consider the system of equations 
y =x +2, 2) =3xy + xz, 


with initial conditions y = 2 and z = O when x = 0. Start with the initial guesses Yo(x) = 2, 
Z,)(x) = 0, use the method of successive approximations, and determine Y3(x) and Z3(x). 
9. Consider the initial-value problem 


y’ =xy'+x4y, with y=5 and y’ =1 whenx =0. 


Change this problem to an equivalent problem involving a system of two equations for two 
unknown functions y = Y(x) and z = Z(x), where z = y’ Then use the method of successive 
approximations, starting with initial guesses Yo(x) = 5 and Z,(x) = 1, and determine Y,(x) 
and Z,(x). 

10. Let f be defined on the rectangle R = [—1, 1] x [-—1, 1] as follows: 


0 if x =0, 

2y/ix if x #0 and |y| <x’, 
f(x, y) = ; ; 

2x if x #O and y >x"*, 


—2x if x #0 and y < —x?. 


(a) Prove that | f(x, y)| <2 for all (x, y) in R. 

(b) Show that f does not satisfy a Lipschitz condition on R. 

(c) For each constant C satisfying |C| < 1, show that y = Cx? is a solution of the initial-value 
problem y =f(x, y), with y = 0 when x =0. Show also that the graph of each of these 
solutions over (—1, 1) lies in R. 

(d) Apply the method of successive approximations to this initial-value problem, starting 
with initial guess Y,)(x) = 0. Determine Y,,(x) and show that the approximations converge to 
a solution of the problem on the interval (—1, 1). 

(ec) Repeat part (d), starting with initial guess Y)(x) = x. Determine Y,,(x) and show that 
the approximating functions converge to a solution different from any of those in part (c). 
(f) Repeat part (d), starting with the initial guess Yo(x) = x°. 

(g) Repeat part (d), starting with the initial guess Yo(x) = xi/3 | 


% 7.25 Successive approximations and fixed points of operators 


The basic idea underlying the method of successive approximations can be used not only 
to establish existence theorems for differential equations but also for many other important 
problems in analysis. The rest of this chapter reformulates the method of successive 
approximations in a setting that greatly increases the scope of its applications. 
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In the proof of Theorem 7.18 we constructed a sequence of functions { Y,} according to 
the recursion formula 


Y,44(x) = B+ [? AY,(t) dt. 


The right-hand member of this formula can be regarded as an operator T which converts 
certain functions Y into new functions 7(Y) according to the equation 


T(Y) =BH+ ii AY(t) dt. 


In the proof of Theorem 7.18 we found that the solution Y of the initial-value problem 
Y(t) = AY(t), Y(a) = B, satisfies the integral equation 


Y=B+ [? ava) dt. 


In operator notation this states that Y = 7(Y). In other words, the solution Y remains 
unaltered by the operator JT. Such a function Y is called a fixed point of the operator T. 

Many important problems in analysis can be formulated so their solution depends on the 
existence of a fixed point for some operator. Therefore it is worthwhile to try to discover 
properties of operators that guarantee the existence of a fixed point. We turn now to a 
systematic treatment of this problem. 


* 7.26 Normed linear spaces 


To formulate the method of successive approximations in a general form it is convenient 
to work within the framework of linear spaces. Let S be an arbitrary linear space. When 
we speak of approximating one element x in S by another element y in S, we consider the 
difference x — y, which we call the error of the approximation. To measure the size of 
this error we introduce a norm in the space. 


DEFINITION OF A NORM. Let S be any linear space. A real-valued function N defined on S 
is called a norm if it has the following properties: 

(a) N(x) > 0 for each x in S. 

(b) N(cx) = |c| N(x) for each x in S and each scalar c. 

(c) N(x + y) < N(x) + N(y) for all x and y in S. 

(d) N(x) = 0 implies x = O. 


A linear space with a norm assigned to it is called a normed linear space. 


The norm of x is sometimes written ||x|| instead of N(x). In this notation the fundamental 
properties become: 

(a) ||x|| > 0 for all x in S. 

(b) ||ex|| = |e] ||x|| for all x in S and all scalars c. 

(c) jx + yl < xl + llyll for all x and y in S. 

(d) ||x|| = 0 implies x = O. 

If x and y are in S, we refer to ||x — y|| as the distance from x to y. 
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If the space S is Euclidean, then it always has a norm which it inherits from the inner 
product, namely, ||x|| = (x, x). However, we shall be interested in a particular norm 
which does not arise from an inner product. 


EXAMPLE. The max norm. Let C(J) denote the linear space of real-valued functions 
continuous on a closed and bounded interval J. If y € C(/), define 


lel = max Ip(x)|, 


where the symbol on the right stands for the maximum absolute value of » on J. The 
reader may verify that this norm has the four fundamental properties. 

The max norm is not derived from an inner product. To prove this we show that the 
max norm violates some property possessed by all inner-product norms. For example, if a 
norm 1s derived from an inner product, then the “parallelogram law”’ 


lx + yl? + |x — yl? = 2 xl? + 2 My? 


holds for all x and yin S. (See Exercise 16 in Section 1.13.) The parallelogram law is not 
always satisfied by the max norm. For example, let x and y be the functions on the interval 
[0, 1] given by 

x) =2; y(t) =1—¢. 


Then we have ||x|| = || y|| = |x + yl] = lx — y|]| = 1, so the parallelogram law is violated. 


* 7.27 Contraction operators 


In this section we consider the normed linear space C(J/) of all real functions continuous 
on a closed bounded interval J in which ||¢|| is the max norm. Consider an operator 


T: CU) > CV) 


whose domain is C(J) and whose range is a subset of C(/). That is, if y is continuous on J, 
then 7(q) is also continuous on J. The following formulas illustrate a few simple examples 
of such operators. In each case is an arbitrary function in C(/) and T(¢)(x) is defined for 
each x in J by the formula given: 


T(p)(x) = Ag(x), where 4 is a fixed real number, 


T(gy\(x) = fe g(t) dt, where c is a given point in J, 
T(px) = b + |* fl, plat, 


where b is a constant and the composition f[t, p(t)] is continuous on J. 

We are interested in those operators T for which the distance ||7(~) — T(y)|| is less than 
a fixed constant multiple « < 1 of ||g — y||. These are called contraction operators; they 
are defined as follows. 
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DEFINITION OF A CONTRACTION OPERATOR. An operator T: C(J)— C(J) is called a 
contraction operator if there is a constant « satisfying 0 < « <1 such that for every pair of 
functions y and py in C(J) we have 
(7.70) IT(p) — Tp) < « lly — yl. 


The constant « is called a contraction constant for T. 


Note: Inequality (7.70) holds if and only if we have 


IT(p)(x) — TY)(~)| < «lle —yll for every x in J. 


EXAMPLE |. Let 7 be the operator defined by 7T(q)(x) = Ag(x), where 2 is constant. 
Since 


IT(P)(x) — TYE)! = IAL 1e@) — vOO, 


we have ||7(y) — T(y)|| = |A| lly — ||. Therefore this operator is a contraction operator 
if and only if |A| < 1, in which case |A| may be used as a contraction constant. 


EXAMPLE 2. Let T(y)(x) = 6 + J*/fIt, p(t] dt, where / satisfies a Lipschitz condition 
of the form 


f(y) —f(% 21S Kly —2| 
for all x in J and all real y and z; here K 1s a positive constant. Let L(J) denote the length 


of the interval J. If KL(J) < 1 we can easily show that T is a contraction operator with 
contraction constant KL(J). In fact, for every x in J we have 


TEx) — TMG = | [* LE o] Ft, wOR dt | < KJ" leo = yoolat 
<K ip — vi)” at| < KLU) Ip — vl. 
If KL(J) < 1, then T is a contraction operator with contraction constant « = KL(J). 


* 7.28 Fixed-point theorem for contraction operators 


The next theorem shows that every contraction operator has a unique fixed point. 


THEOREM 7.20. Let T: C(J)—> C(J) be a contraction operator. Then there exists one and 
only one function p in C(J) such that 


(7.71) T(¢) = ¢. 


Proof. Let op be any function in C(/) and define a sequence of functions {y,} by the 
recursion formula 


Pnii=T(y,) for n=0,1,2,.... 
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Note that ,,,, € C(J) for each n. We shall prove that the sequence {@,,} converges to a 


limit function g in C(/). The method is similar to that used in the proof of Theorem 7.18. 
We write each ¢, as a telescoping sum, 


(7.72) paAlX) = gol) +> {nn — 99} 


and prove convergence of {y,} by showing that the infinite series 


(7.73) glx) + ¥ {en() — 94(2)} 


converges uniformly on J. Then we show that the sum of this series is the required fixed 
point. 

The uniform convergence of the series will be established by comparing it with the 
convergent geometric series 


M > a* : 
k=0 
where M = ||qo|| + ||q||, and « is a contraction constant for 7. The comparison is 
provided by the inequality 
(7.74) lPrra(~) — %(x)| < Ma* 


which holds for every x in J and every k > 1. To prove (7.74) we note that 
| Perr) — POD = ITCPIOC) — TEPrDO) S & Pe — Peall- 

Therefore the inequality in (7.74) will be proved if we show that 

(7.79) lex — Pr-all << Mot 

for every kK > 1. We now prove (7.75) by induction. For k = | we have 


IP: -— Poll S Igall + Ieoll = M, 


which is the same as (7.75). To prove that (7.75) holds for k + 1 if it holds for k we note 
that 


lPeirx) — Pex) = |T(Gx)(%) — Teed) < & Wee — Gp-all < Me*. 
Since this is valid for each x in J we must also have 
Il Peta — || < Ma*. 


This proves (7.75) by induction. Therefore the series in (7.73) converges uniformly on J. 
If we let w(x) denote its sum we have 


(7.76) wx) = lim gC) = PuC%) +E {Pesal) ~ P40}. 
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The function ¢ is continuous on J because it is the sum of a uniformly convergent series 


of continuous functions. To prove that @ is a fixed point of T we compare 7(q) with 
Pnii = T(p,). Using the contraction property of T we have 


(7.77) IT(P)(X) — Pav) = IT(P)C) — TEPID! <S «1P) — 9, (DI. 


But from (7.72) and (7.76) we find 


1) = PG) =] Sern) — 149} | SE lpersle) — eI < MEA, 


where in the last step we used (7.74). Therefore (7.77) implies 


IT) = Pra S MDa, 


When n — oo the series on the right tends to 0, so »,,,;(x) > T()(x). But since ¢,,,(x) > 


g(x) as n—> oO, this proves that y(x) = T(¢)(x) for each x in J. Therefore y = 7(9), 
So @ is a fixed point. 


Finally we prove that the fixed point is unique. Let py be another function in C(J) such 
that 7(yw) = y. Then we have 


le — vll = IT) — TH) S@ Iy — yl. 


This gives us (1 — «) |g — pl] < 0. Since « < 1 we may divide by 1 — « to obtain the 
inequality ||g¢ — y|| < 0. But since we also have ||g — y|| > 0 this means that ||y — y|| = 
0, and hence g — y = 0. The proof of the fixed-point theorem is now complete. 


* 7.29 Applications of the fixed-point theorem 


To indicate the broad scope of applications of the fixed point theorem we use it to prove 
two important theorems. The first gives a sufficient condition for an equation of the form 
I(x, y) = 0 to define y as a function of x. 


THEOREM 7.21. AN IMPLICIT-FUNCTION THEOREM. Let f be defined on a rectangular strip 
R of the form 
R={, y)lagxgb,-w<y< +o}. 


Assume that the partial derivative D, f(x, y) existst and satisfies an inequality of the form 


(7.78) 0<m< Def(x,y) <M 


+ Df (x, y) is the derivative of f(x, y) with respect to , holding x fixed. 


238 Systems of differential equations 


for all (x, y) in R, where m and M are constants withm < M. Assume also that for each 
function y continuous on [a,b] the composite function g(x) = f[x, 9(x)] is continuous on 
[a, b]. Then there exists one and only one function y = Y(x), continuous on [a, b], such that 


(7.79) fix, Y(x)] = 0 


for all x in {a, b]. 


Note: We describe this result by saying that the equation f(x, y) = 0 serves to define 
y implicitly as a function of x in [a, 5]. 


Proof. Let C denote the linear space of continuous functions on [a, b], and define an 
operator T: C — C by the equation 


T(o\x) = ox) — _ fis, 9(x)] 


for each x in [a,b]. Here M is the positive constant in (7.78). The function T(g) eC 
whenever y € C. We shall prove that T is a contraction operator. Once we know this it 
follows that T has a unique fixed point Yin C. For this function Y we have Y = T(Y) 
which means 


1270 = _ fis, ¥(x)] 


for each x in [a, b]. This gives us (7.79), as required. 
To show that TJ is a contraction operator we consider the difference 


_ Sh, 7(x)] = Fx, y(x)] 


(7.80) T( p(x) — T(yx) = 9x) — v(x) a 


By the mean-value theorem for derivatives we have 
fix, pO)] — fix, ypO)] = Deflx, zE)II9@) — vd], 


where z(x) lies between g(x) and p(x). Therefore (7.80) gives us 


(7.81) T( px) — T(y\(x) = [9% — vOI( = eel | 
The hypothesis (7.78) implies that 
O1:— Df [x, 20)] <1- JU 
M M 
Therefore (7.81) gives us the inequality 
(7.82) ITP) — TIP)! < le) — vO! ( 7 a < alg — yl, 


where « = 1 — m/M. Since 0 <m< M, we have0 < «<1. Inequality (7.82) is valid 
for every x in [a, b]. Hence T is a contraction operator. This completes the proof. 
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The next application of the fixed-point theorem establishes an existence theorem for the 
integral equation 


(7.83) p(x) = lx) + 2]? K(x, Dg(0) dt. 


Here y is a given function, continuous on [a, b], A is a given constant, and K is a given 
function defined and bounded on the square 


S={(x,y|acxgbagcy<b)d}. 


The function K is called the kernel of the integral equation. Let C be the linear space of 
continuous functions on [a, b]. We assume that the kernel K is such that the operator T 
given by 
b 
T(g)(x) = wx) +4? Kx, Not) at 


maps C into C. In other words, we assume that 7(~) € C whenever py € C. A solution of 
the integral equation is any function g in C that satisfies (7.83). 


THEOREM 7.22. AN EXISTENCE THEOREM FOR INTEGRAL EQUATIONS. Jf, under the foregoing 
conditions, we have 


(7.84) K(x, y)| <M 
for all (x, y) in S, where M > 0, then for each A such that 


1 
7.85 A| << —-——_ 

(7.85) tea 
there is one and only one function in C that satisfies the integral equation (7.83). 


Proof. We shall prove that T is a contraction operator. Take any two functions ¢, and 
@, in C and consider the difference 


T(p(x) — Tg) = AJ" Kx, Dipl) — gl] at 
Using the inequality (7.84) we may write 
IT p(x) — T(p2)(X)| < [Al MO — 2) yi — Gall = lor — Gall, 


where « = |A| M(b — a). Because of (7.85) we have 0 < « <1, so T is a contraction 
operator with contraction constant «. Therefore T has a unique fixed point in C. This 
function 9 satisfies (7.83). 


PART 2 


NONLINEAR ANALYSIS 


8 


DIFFERENTIAL CALCULUS OF SCALAR AND 
VECTOR FIELDS 


8.1 Functions from R” to R™. Scalar and vector fields 


Part 1 of this volume dealt primarily with linear transformations 
T:V—>W 


from one linear space V into another linear space W. In Part 2 we drop the requirement 
that 7 be linear but restrict the spaces V and W to be finite-dimensional. Specifically, we 
shall consider functions with domain in n-space R” and with range in m-space R™. 

When both n and m are equal to 1, such a function is called a real-valued function of a 
real variable. Whenn = 1 andm > 1 it is called a vector-valued function of a real variable. 
Examples of such functions were studied extensively in Volume I. 

In this chapter we assume thatn > 1 andm > 1. When m = | the function is called a 
real-valued function of a vector variable or, more briefly, a scalar field. When m > 1 it is 
called a vector-valued function of a vector variable, or simply a vector field. 

This chapter extends the concepts of limit, continuity, and derivative to scalar and 
vector fields. Chapters 10 and 11 extend the concept of the integral. 


Notation: Scalars will be denoted by light-faced type, and vectors by bold-faced type. 
If f is a scalar field defined at a point x = (x,,...,x,) in R”, the notations f(x) and 
I(x1,...,X,) are both used to denote the value of fat that particular point. If fis a vector 
field we write f(x) or f(x,,...,.X,) for the function value at x. We shall use the inner 


product 
XY =DXYe 
k=1 


and the corresponding norm ||x|| = (x-x)*, where x =(x1,...,X,) and y= 
(\1,--+>5 Yn). Points in R? are usually denoted by (x, y) instead of (x,, x2); points in R® 
by (x, y, z) instead of (x1, X2, X3). 

Scalar and vector fields defined on subsets of R? and R® occur frequently in the appli- 
cations of mathematics to science and engineering. For example, if at each point x of the 
atmosphere we assign a real number f(x) which represents the temperature at x, the function 
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f so defined is a scalar field. If we assign a vector which represents the wind velocity at 
that point, we obtain an example of a vector field. 

In physical problems dealing with either scalar or vector fields it is important to know 
how the field changes as we move from one point to another. In the one-dimensional case 
the derivative is the mathematical tool used to study such changes. Derivative theory in 
the one-dimensional case deals with functions defined on open intervals. To extend the 
theory to R” we consider generalizations of open intervals called open sets. 


8.2 Open balls and open sets 


Let a be a given point in R” and let r be a given positive number. The set of all points x 
in R” such that 


|x —al <r 


is called an open n-ball of radius r and center a. We denote this set by B(a) or by B(a; r). 

The ball B(a; r) consists of all points whose distance from a is less than r. In R! this is 
simply an open interval with center at a. In R? it is a circular disk, and in R? it is a spherical 
solid with center at a and radius r. 


DEFINITION OF AN INTERIOR POINT. Let S be a subset of R”, and assume thatae S. Then 
a is called an interior point of S if there is an open n-ball with center at a, all of whose points 
belong to S. 


In other words, every interior point a of S can be surrounded by an n-ball B(a) such that 
B(a) < S. The set of all interior points of S is called the interior of S and is denoted by 
int S. An open set containing a point a is sometimes called a neighborhood of a. 


DEFINITION OF AN OPEN SET. A Set S in R” is called open if all its points are interior points. 
In other words, S is open if and only if S = int S. 


EXAMPLES. In R! the simplest type of open set is an open interval. The union of two or 
more open intervals is also open. A closed interval [a, 6] is not an open set because neither 
endpoint of the interval can be enclosed in a 1-ball lying within the given interval. 

The 2-ball S = B(O; 1) shown in Figure 8.1 is an example of an open set in R*. Every 
point a of S is the center of a disk lying entirely in S. For some points the radius of this 
disk is very small. 

Some open sets in R? can be constructed by taking the Cartesian product of open sets in 
R!. If A, and A, are subsets of R!, their Cartesian product A, x Ag, is the set in R? defined 
by 

A, X Ap = {(@,, 4)|a,€A, and a,€ A}. 


An example is shown in Figure 8.2. The sets A, and A, are intervals, and A, x A, is a 
rectangle. 

If A, and A, are open subsets of R', then A, X A, will be an open subset of R*?. To prove 
this, choose any point a = (a), a.) in A; X Ag. We must show that a is an interior point 
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Circular disk 
FiGureE 8.1 The disk B(O;1) is an open FiGuRE 8.2 The Cartesian product of two 
set in R?. open intervals is an open rectangle. 


of A, x A,. Since A, and A, are open in R' there is a 1-ball B(a,; r,) in A, and a 1-ball 
B(a,;1r.) in A,. Let r = min {r,,r.}. We can easily show that the 2-ball Bia;r)¢ 
A, X Ag. In fact, ifx = (x,, x2) is any point of B(a; r) then ||x — al| << r,so|x, —a,| <r, 
and |x, —4a,|<r,. Hence x,€ B(a,;r,) and x,€ B(a,;r,). Therefore x,¢A, and 
X_ € Ay, SO (X1,X2)€ A, X Ag. This proves that every point of B(a;r) is in A, X Ag. 
Therefore every point of A, xX Ag is an interior point, so A, X A, IS open. 


The reader should realize that an open subset of R’ is no longer an open set when it is 
considered as a subset of R?, because a subset of R! cannot contain a 2-ball. 


DEFINITIONS OF EXTERIOR AND BOUNDARY. A point x is said to be exterior to a set S in 
R” if there is an n-ball B(x) containing no points of S. The set of all points in R" exterior to S 
is called the exterior of S and is denoted by ext S. A point which is neither exterior to S nor an 
interior point of S is called a boundary point of S. The set of all boundary points of S is called 
the boundary of S and is denoted by OS. 


These concepts are illustrated in Figure 8.1. The exterior of S is the set of all x with 
|x|] > 1. The boundary of S consists of all x with ||x|| = 1. 


8.3. Exercises 


1. Let f be a scalar field defined on a set S and let c be a given real number. The set of all points 
x in S such that f(x) = c is called a /evel set of f- (Geometric and physical problems dealing 
with level sets will be discussed later in this chapter.) For each of the following scalar fields, 
S is the whole space R". Make a sketch to describe the level sets corresponding to the given 
values of c. 


(a) f(x,y) =x? + y’, c =0,1,4,9. 

(b) f(x, y) = e””, c =e, e7), 1, e, e, e. 
(c) f(x, y) = cos (x + y), c = —1,0,4, 42,1 
(d) fX,y,z) =x +y +z, c= —1,0,1. 

(e) f(x, y, z) = x? + 2y? + 32%, c = 0, 6, 12 

(f) f(x, y, z) = sin (x? + y? + rg) c= = —4, 0,3,/2, I 
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2. In each of the following cases, let S be the set of all points (x, y) in the plane satisfying the 
given inequalities. Make a sketch showing the set S and explain, by a geometric argument, 
whether or not S is open. Indicate the boundary of S on your sketch. 


(a) x2 +y? <1. (h) 1 <x <2and3 <y <4. 

(b) 3x? + 2y? < 6. Gi) 1<x<2andy>0. 

(c) |x| <1 andly| <1. (j) x2y. 

(d) x >Oandy>0. (k) x >y. 

(e) |x| < land |ly| <1. (l) y > x? and |x| <2. 

(f) x >Oandy <0. (m) (x? + y? — 1)(4 — x? — y’) > 0. 
(g) xy <1. (n) (2x — x? — y*)(x? + y? — x) > 0. 


3. In each of the following, let S be the set of all points (x, y, z) in 3-space satisfying the given 
inequalities and determine whether or not S is open. 
(a) 22 —x* —-y-1>0. 
(b) |x| <1, |yl <1, and [z| <1. 
(c) x ty +z<1l. 
(d) |x| <1, lyl <1, and |z| <1. 
(ce) x ty +z<landx >0,y>0,z>0. 
(f) x? + 4y? + 427 — 2x + 16y + 40z + 113 <0. 

4. (a) If A is an open set in n-space and if x€ A, show that the set A — {x}, obtained by 
removing the point x from A, is open. 
(b) If A is an open interval on the real line and B is a closed subinterval of A, show that 
A — Bis open.t 
(c) If A and B are open intervals on the real line, show that A U Band A € B are open. 
(d) If A is a closed interval on the real line, show that its complement (relative to the whole 
real line) is open. 

5. Prove the following properties of open sets in R”: 
(a) The empty set @ is open. 
(b) R” is open. 
(c) The union of any collection of open sets is open. 
(d) The intersection of a finite collection of open sets is open. 
(e) Give an example to show that the intersection of an infinite collection of open sets is not 
necessarily open. 


Closed sets. A set Sin R” is called closed if its complement R” — Sis open. The next three 
exercises discuss properties of closed sets. 


6. In each of the following cases, let S be the set of all points (x, y) in R? satisfying the given 
conditions. Make a sketch showing the set S and give a geometric argument to explain whether 
Sis open, closed, both open and closed, or neither open nor closed. 


(a) x? +y?>0. (g) 1 <x <2,3 cy <4. 
(b) x? + y? <0. (hy) 1 <x <2,3<y <4. 
(c) P+ y? <1. (i) y = x?. 

(d)1 <x? +y? <2. Gj) y= x. 

(e) lox? +y? <2. (k) y > x* and |x| <2. 
(f) 1<x+y? <2. (1) y > x* and |x| <2. 


7. (a) If A is a closed set in n-space and x is a point not in A, prove that A U {x} is also closed. 
(b) Prove that a closed interval [a, b] on the real line is a closed set. 
(c) If A and B are closed intervals on the real line, show that A U Band A €O B are closed. 


+ If A and B are sets, the difference A — B (called the complement of B relative to A) is the set of all 
elements of A which are not in B. 
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8. Prove the following properties of closed sets in R”. You may use the results of Exercise 5. 
(a) The empty set 2 is closed. 
(b) R” is closed. 
(c) The intersection of any collection of closed sets is closed. 
(d) The union of a finite number of closed sets is closed. 
(e) Give an example to show that the union of an infinite collection of closed sets is not 
necessarily closed. 

9. Let S be a subset of R”. 
(a) Prove that both int S and ext S are open sets. 
(b) Prove that R” = (int S) U (ext S) U OS, a union of disjoint sets, and use this to deduce 
that the boundary @S is always a closed set. 

10. Given a set S in R” and a point x with the property that every ball B(x) contains both interior 
points of S and points exterior to S. Prove that x is a boundary point of S. Is the converse 
statement true? That is, does every boundary point of S necessarily have this property? 

11. Let S be a subset of R”. Prove that ext S = int(R” — S). 

12. Prove that a set S in R” is closed if and only if S = (int S) U aS. 


8.4 Limits and continuity 


The concepts of limit and continuity are easily extended to scalar and vector fields. We 
shall formulate the definitions for vector fields; they apply also to scalar fields. 

We consider a function f: S > R™, where S is a subset of R". If ae R” and b€ R™ we 
write 


(8.1) lim f(x) = b (or, f(x) > bas x >a) 
to mean that 
(8.2) Pee | f(x) — bi = 0. 


The limit symbol in equation (8.2) is the usual limit of elementary calculus. In this definition 
it is not required that f be defined at the point a itself. 
If we write h = x — a, Equation (8.2) becomes 


lim || f(a + A) — b|| = 0. 
||| +0 


For points in R? we write (x, y) for x and (a, b) for a and express the limit relation (8.1) as 


follows: 
lim f(x, y)=b). 


(x,y) (a,b) 


For points in R® we put x = (x, y, z) and a = (a, b, c) and write 


lim f(x, y, Zz) = 5. 


(x,y,z) > (a,b,c) 


A function f is said to be continuous at a if f is defined at a and if 
lim f(x) = f(a). 


We say f is continuous on a set S if f is continuous at each point of S. 
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Since these definitions are straightforward extensions of those in the one-dimensional 
case, it is not surprising to learn that many familiar properties of limits and continuity can 
also be extended. For example, the usual theorems concerning limits and continuity of 
sums, products, and quotients also hold for scalar fields. For vector fields, quotients are 
not defined but we have the following theorem concerning sums, multiplication by scalars, 
inner products, and norms. 


THEOREM 8.1. Jf lim f(x) = b and lim g(x) =, then we also have: 


(a) lim [f(x) + s@)] = 5 +e. 
(b) lim Af(x) = Ab for every scalar i. 


(c) lim f(x) g(x) = be. 
(d) lim | f(x)|| = oI. 


Proof. We prove only parts (c) and (d); proofs of (a) and (b) are left as exercises for 
the reader. 
To prove (c) we write 


f(x): g(x) — b+ ¢ = [flx) — 5) - [g(x) — €] + 5: [g(x) — €] + - f(x) — 4). 
Now we use the triangle inequality and the Cauchy-Schwarz inequality to obtain 
0 < IIflx)- a(x) — bell < Iflx) — BI lg) — ell + [dT Ilg@) — ell + fell If) — ll. 
Since || f(x) — b|| — 0 and ||g(x) — c|| — 0 as x — a, this shows that ||f(x) - g(x) — b+ ell > 


0 as x a, which proves (c). 
Taking f(x) = g(x) in part (c) we find 


lim || f(x)II" = 61", 
from which we obtain (d). 


EXAMPLE 1. Continuity and components of a vector field. If a vector field f has values in 
R™, each function value f(x) has m components and we can write 


F(x) = Ai), «+ sm). 


The m scalar fields f,,...,/,, are called components of the vector field f/ We shall prove 
that fis continuous at a point if, and only if, each component f, is continuous at that point. 

Let e, denote the Ath unit coordinate vector (all the components of e, are 0 except the 
kth, which is equal to 1). Then /,(x) is given by the dot product 


fi, (x) = f(x) * &- 
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Therefore, part (c) of Theorem 8.1 shows that each point of continuity of fis also a point of 
continuity of f,. Moreover, since we have 


f(x) df W(X)Ez » 


repeated application of parts (a) and (b) of Theorem 8.1 shows that a point of continuity 
of all m components f,,...,/, is also a point of continuity of f. 


EXAMPLE 2. Continuity of the identity function. The identity function, f(x) =x, is 
continuous everywhere in R". Therefore its components are also continuous everywhere 
in R”. These are the n scalar fields given by 


fi(x) = X15 fo(x) = Xar-6- »Sn(X) = Xn- 


EXAMPLE 3. Continuity of linear transformations. Let f: R” — R” be a linear transforma- 
tion. We will prove that fis continuous at each point ain R”. By linearity we have 


f(a + h) = fla) + fl). 
Therefore, it suffices to prove that f(h) — Oash— O. Writing hin terms of its components 
we have h= he, +°:: +h,e,. Using linearity again we find f(A) = h,f(e,) +-°° + 
h, f(e,,). This shows that f(h) ~ Oash—O. 


EXAMPLE 4. Continuity of polynomials in n variables. A scalar field P defined on R” by a 
formula of the form 


P1 Pn k k 
P(x) — > eee > Cry : ‘ItnX1 eee bt 
k,=0 kn=0 


is called a polynomial in n variables x,,...,x,. A polynomial is continuous everywhere 
in R” because it is a finite sum of products of scalar fields continuous everywhere in R”- 
For example, a polynomial in two variables x and y, given by 


p qa a 
P(x, y) = Da > Ci3X'y’ 


is continuous at every point (x, y) in R®. 


EXAMPLE 5. Continuity of rational functions. A scalar field f given by f(x) = P(x)/Q(x), 
where P and Q are polynomials in the components of x, is called a rational function. A 
rational function is continuous at each point where Q(x) ¥ 0. 


Further examples of continuous function can be constructed with the help of the next 
theorem, which is concerned with continuity of composite functions. 
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THEOREM 8.2. Let fand g be functions such that the composite function f © g is defined at 
a, where 


(f° g)(x) = fl[g(x)]. 


If g is continuous at a and if f is continuous at g(a), then the composition f ° g is continuous 
ata. 


Proof. Let y = f(x) and b = g(a). Then we have 


flg(x)] — flg(a)] = f(y) — f(S). 


By hypothesis, y — b as x + a, so we have 


lim || flg(x)] — fls(@)l = , tim Ifo) — f(b)|| = 9. 


\|¥—a||—0 


Therefore lim f[g(x)] = f[g(a)], so f° g is continuous at a. 


EXAMPLE 6. The foregoing theorem implies continuity of scalar fields h, where h(x, y) is 
given by formulas such as 


e+y 


sin(x’y), log (x? + y’), = 5? loa [eos (x? + y>)). 


These examples are continuous at all points at which the functions are defined. The first is 
continuous at all points in the plane, and the second at all points except the origin. The 
third is continuous at all points (x, y) at which x + y ¥ 0, and the fourth at all points at 
which x? + y? is not an odd multiple of 7/2. [The set of (x, y) such that x? + y? = nz/2, 
n=1,3,5,..., is a family of circles centered at the origin.] These examples show that 
the discontinuities of a function of two variables may consist of isolated points, entire 
curves, or families of curves. 


EXAMPLE 7. A function of two variables may be continuous in each variable separately 
and yet be discontinuous as a function of the two variables together. This is illustrated by 
the following example: 


fxy= =", if &y)F@O, f(0,0) =0. 
x" + y 


For points (x, y) on the x-axis we have y = 0 and f(x, y) = f(x, 0) = 0, so the function 
has the constant value 0 everywhere on the x-axis. Therefore, if we put y = 0 and think of 
fas a function of x alone, fis continuous at x = 0. Similarly, fhas the constant value 0 
at all points on the y-axis, so if we put x = 0 and think of f as a function of y alone, fis 
continuous at y = 0. However, as a function of two variables, fis not continuous at the 
origin. In fact, at each point of the line y = x (except at the origin) the function has the 
constant value 4 because f(x, x) = x*/(2x”) = 4; since there are points on this line 
arbitrarily close to the origin and since f(0, 0) ¥ 3, the function is not continuous at (0, 0). 
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8.5 Exercises 


The exercises in this section are concerned with limits and continuity of scalar fields defined on 


subsets of the plane. 


1. 


In each of the following examples a scalar field fis defined by the given equation for all points 
(x, y) in the plane for which the expression on the right is defined. In each example determine 
the set of points (x, y) at which fis continuous. 


(a) f(x,y) = xt + yt — 4x2y2, (f) “f(x, y) = arcsin ——— . 
JP +y 

(b) f(x, y) = log (x? + y?). (g) f(x, y) = arctan a 

On Cee (h) f(x, y) 

x, = B, See xX, =e 
LOY y z oe 52 
2 
(d) f(x,y) = tan (i) f(x,y) = x0, 
y e 


(e) f(x, y) = arctan (j) f(x, y) = arccos J/xly. 


x 


If lim f(x, y) = ZL, and if the one-dimensional limits 


(x,y)—(a,b) 


lim f(x, y) and lim f(x, y) 
x y—>b 


—>a 


both exist, prove that 
lim [lim f(x, y)] = lim [lim f(x, y)] = L. 


2a yobd y>b xa 


The two limits in this equation are called iterated limits; the exercise shows that the existence of 
the two-dimensional limit and of the two one-dimensional limits implies the existence and equality 
of the two iterated limits. (The converse is not always true. A counter example is given in 
Exercise 4.) 


. Let f(x, y) = (x — y)/(x + y) if x + y #0. Show that 


lim [lim f(x, y)] =1 but that lim [lim f(x, y)] = -1. 


a«->0 y—0 yO 2x0 


Use this result with Exercise 2 to deduce that f(x, y) does not tend to a limit as (x, y) — (0, 0). 
Let 


2 


eee eee 
x®y? + (x — y)? 


2 


f(x,y) = whenever x?y? + (x — y)? #0. 


Show that 
lim [lim f(x, y)] = lim [lim f(x, y)] = 0 


z—0 y—0 y—0 «2-0 


but that f(x, y) does not tend to a limit as (x, y) — (0, 0). [Hint: Examine fon the line y = x.] 
This example shows that the converse of Exercise 2 is not always true. 
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5. Let 
1 
x sin - if y #0, 
f(x, y) = - 
0 if y=0. 


Show that f(x, y) > 0 as (x, y) — (0, 0) but that 


lim [lim f(x, y)] # lim [lim f(x, y)]. 
y>0 x0 z—>0 y>0 
Explain why this does not contradict Exercise 2. 

6. If (x, y) ¥ (0, 0), let f(x, y) = (x? — y®)/(x? + y’). Find the limit of f(x, y) as (x, y) > (0, 0) 
along the line y = mx. Is it possible to define f(0, 0) so as to make fcontinuous at (0, 0)? 

7. Let f(x, y) = Oif y < Oor if y > x? and let f(x, y) = 1 if 0 < y < x?. Show that f(x, y) +0 
as (x, y) > (0, 0) along any straight line through the origin. Find a curve through the origin 
along which (except at the origin) f(x, y) has the constant-value 1. Is fcontinuous at the origin? 

8. If f(x, y) = [sin (x? + y*)]/(x? + y?) when (x, y) ¥ (0,0) how must f(0, 0) be defined so as 
to make f continuous at the origin? 

9. Let fbe a scalar field continuous at an interior point a of a set Sin R”. If f(a) #0, prove that 
there is an n-ball B(a) in which f has the same sign as f(a). 


8.6 The derivative of a scalar field with respect to a vector 


This section introduces derivatives of scalar fields. Derivatives of vector fields are dis- 
cussed in Section 8.18. 

Let f be a scalar field defined on a set S in R”, and let a be an interior point of S. We 
wish to study how the field changes as we move from a to a nearby point. For example, 
suppose f(a) is the temperature at a point a in a heated room with an open window. If we 
move toward the window the temperature will decrease, but if we move toward the heater 
it will increase. In general, the manner in which a field changes will depend on the direction 
in which we move away from a. 

Suppose we specify this direction by another vector y. That is, suppose we move from a 
toward another point a + y along the line segment joining aand a + y. Each point on this 
segment has the form a + hy, where h is a real number. An example is shown in Figure 
8.3. The distance from ato a + hy is ||Ay|| = [Al |ly|. 

Since a is an interior point of S, there is an -ball B(a; r) lying entirely in S. If Ais chosen 
so that |h| ||y|| <r, the segment from ato a + hy will lie in S. (See Figure 8.4.) We keep 


a+hy 
a 
a+hy 
y 
Siew 
FiGure 8.3 The point a + hy lies on the line FiGure 8.4 The point a + hy lies in the 


through a parallel to y. n-ball B(a; r) if ||hy|| <r. 
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h # 0 but small enough to guarantee that a + hy € S and we form the difference quotient 


(8.3) flat+ 0 — f(a) . 


The numerator of this quotient tells us how much the function changes when we move 
from atoa-+hy. The quotient itself is called the average rate of change of f over the line 
segment joining ato a+ hy. We are interested in the behavior of this quotient as h ~ 0. 
This leads us to the following definition. 


DEFINITION OF THE DERIVATIVE OF A SCALAR FIELD WITH RESPECT TO A VECTOR. Given a 
scalar field f: S—> R, where S < R”. Let abe an interior point of S and let y be an arbitrary 


point inR”. The derivative of f at a with respect to y is denoted by the symbol f'(a; y) and is 
defined by the equation 


(84) f'(aiy) = time +) — 


when the limit on the right exists. 


EXAMPLE |. If y = O, the difference quotient (8.3) is 0 for every h #0, so f’(a; O) 
always exists and equals 0. 


EXAMPLE 2. Derivative of a linear transformation. If f: S — Ris linear, then f(a + hy) = 
f(a) + hf(y) and the difference quotient (8.3) is equal to f(y) for every h ~ 0. In this case, 
f(a; y) always exists and is given by 


f(a; y) = f(y) 


for every ain S and every yin R”. In other words, the derivative of a linear transformation 
with respect to y is equal to the value of the function at y. 


To study how f behaves on the line passing through a and a + y for y ¥ O we introduce 
the function 


e(t)h=f(at+ yy). 


The next theorem relates the derivatives g’(t) and f’(a + ty; y). 


THEOREM 8.3. Let g(t) =f(a+ ty). If one of the derivatives g(t) or f'(a + ty; y) 
exists then the other also exists and the two are equal, 


(8.5) g(th=f‘(at+ wy). 


In particular, when t = 0 we have g'(0) = f'(a; y). 
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Proof. Forming the difference quotient for g, we have 


g(t +h) — g(t) _ fla + ty + hy) — f(a + ty) 
h h 


Letting h — 0 we obtain (8.5). 
EXAMPLE 3. Compute f’(a; y) if f(x) = ||x||? for all x in R”. 


Solution. We let g(t)=f(at+ty) =(a+y):(@atty) =a-at+2ta:ytry-y. 
Therefore 2’(t) = 2a-y + 2ty-y,sog'(0) =f'(a; y) =2a-y. 


A simple corollary of Theorem 8.3 is the mean-value theorem for scalar fields. 


THEOREM 8.4. MEAN-VALUE THEOREM FOR DERIVATIVES OF SCALAR FIELDS. Assume the 
derivative f'(a + ty; y) exists for each t in the interval0 < t <1. Then for some real 6 in 
the open interval) < 6 <1 we have 


faty)—-f@ =f'(z;y), where z=a- Oy. 


Proof. Let g(t) = f(a + ty). Applying the one-dimensional mean-value theorem to g 
on the interval [0, 1] we have 


g(1) — g(0) = g'(9), where 0<6< 1. 


Since g(1) — g(0) = f(a + y) — f(@ and g'(6) = f(a + 4y; y), this completes the proof. 


8.7 Directional derivatives and partial derivatives 


In the special case when y is a unit vector, that is, when ||y|| = 1, the distance between a 
and a + hy is |h|. In this case the difference quotient (8.3) represents the average rate of 
change of f per unit distance along the segment joining a to a + hy; the derivative f’(a; y) 
is called a directional derivative. 


DEFINITION OF DIRECTIONAL AND PARTIAL DERIVATIVES. Jf y is a unit vector, the derivative 
J'(a; y) is called the directional derivative of f at a in the direction of y. In particular, if 
y =e, (the kth unit coordinate vector) the directional derivative f'(a; e,) is called the partial 
derivative with respect to e,, and is also denoted by the symbol D,.f(a). Thus, 


D,f(a@) =f" (a; &)- 
The following notations are also used for the partial derivative D, f(a): 


) / 
D,f (ay, ..+54n)s ieee itd and sf, (@,,. + +5 n)- 
k 
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Sometimes the derivative f, is written without the prime as f, or even more simply 
as f,. 

In R? the unit coordinate vectors are denoted by iandj. Ifa = (a, b) the partial derivatives 
Jf '(a; i) and f'(a; 7) are also written as 


Fab) and X(a,by, 

Ox dy 
respectively. In R°, if a = (a, b, c) the partial derivatives D, f(a), D, f(a), and D3; f(a) are 
also denoted by 


Aa (a, b,c), Ge (a, b,c), and of (a, b,c). 
Ox Oy Oz 


8.8 Partial derivatives of higher order 


Partial differentiation produces new scalar fields D, f,..., D,f from a given scalar field 
f. The partial derivatives of D,f,..., D, fare called second-order partial derivatives of f. 
For functions of two variables there are four second-order partial derivatives, which are 
written as follows: 


3° 0° 3° Q” 
D,(D,f) = “ ’ D,(D2f) = ay * ’ D{Dj,f ) = ae ’ D(D2f) = a . 


Note that D,(D,f) means the partial derivative of D,f with respect to the first variable. 
We sometimes use the notation D; ,f for the second-order partial derivative D;(D,f). For 
example, D,»f= D,(D2f). In the d-notation we indicate the order of derivatives by 
writing 


539 — axl) 
dxdy dax\dy)— 


This may or may not be equal to the other mixed partial derivative, 


a= a(S) 
dydax  dy\dax)— 


In Section 8.23 we shall prove that the two mixed partials D,(D,f) and D,(D,/f) are equal 
at a point if one of them is continuous in a neighborhood of the point. Section 8.23 also 
contains an example in which D,(D,f) # D,(D,f) at a point. 


8.9 Exercises 


1. A scalar field f is defined on R” by the equation f(x) = a-x, where a is a constant vector. 
Compute f ‘(x; y) for arbitrary x and y. 
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2. (a) Solve Exercise 1 when f(x) = ||x||*. 
(b) Take n = 2 in (a) and find all points (x, y) for which f’(2i + 3; xi + yj) =6. 
(c) Taken = 3 in (a)and find all points (x, y, z) for which f’(i + 2j + 3k; xi + yj + zk) =0. 
3. Let T: R” +R” be a given linear transformation. Compute the derivative f’(x; y) for the 
scalar field defined on R” by the equation f(x) = x- T(x). 


In each of Exercises 4 through 9, compute all first-order partial derivatives of the given scalar field. 
The fields in Exercises 8 and 9 are defined on R”. 


+ 
4. f(x,y) =x? + y® sin (xy). 7. fly) = xe Ys 
5. f(x,y) = Sx? + y?. 8. f(x) =a-x, a fixed. 
x n 
6. f(x y) = = s,s @ y) 0,0). 9. f(x) = > Dd ayjxix;, ayy = az. 
es +y t=1 j=1 


In each of Exercises 10 through 17, compute all first-order partial derivatives. In each of Exercises 
10, 11, and 12 verify that the mixed partials D,(D,f) and D,(D, f) are equal. 


10. f(x, y) = x4 + y* — 4x*y?. 14. f(x, y) = arctan (y/x), x £0. 


+ 
11. f(x,y) = log? +9), y) # (0,0). 15. f(x, y) = arctan — = xy #1, 
12. f(x, y) ro x®, yp #0. 16. f(x,y) =x, =x >0. 


13. f(x, y) = tan (x?/y), y #0. 17. f(x, y) = arccos /xly, y £0. 


18. Let v(r, t) =t"e-"/(40), Find a value of the constant 1 such that v satisfies the following 
equation: 


19. Given z = u(x, y)e**+>4 and 07u/(0x dy) = 0. Find values of the constants a and b such that 


20. (a) Assume that f ‘(x; y) = 0 for every x in some n-ball B(a) and for every vector y. Use the 
mean-value theorem to prove that fis constant on B(a). 
(b) Suppose that f ‘(x; y) =O fora fixed vector y and for every x in B(a). What can you 
conclude about fin this case? 

21. A set S in R” is called convex if for every pair of points a and b in S the line segment from a 
to bis also in S; in other words, ta + (1 — t)b€ S for each ¢ in the intervalO <¢ <1. 
(a) Prove that every n-ball is convex. 
(b) If f’(x; y) = 0 for every x in an open convex set S and for every y in R”, prove that fis 
constant on S. 

22. (a) Prove that there is no scalar field f such that f’(a; y) > 0 for a fixed vector a and every 
nonzero vector y. 
(b) Give an example of a scalar field f such that f’(x; y) > 0 for a fixed vector y and every 
vector x. 
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8.10 Directional derivatives and continuity 


In the one-dimensional theory, existence of the derivative of a function f at a point 
implies continuity at that point. This is easily proved by choosing an h ¥ 0 and writing 


fla+h)—f(@. h. 


fath)-—f@= h 


As h-~ 0 the right side tends to the limit f’(a) -0 = 0 and hence f(a + h) > f(a). This 
shows that the existence of f’(a) implies continuity of f at a. 

Suppose we apply the same argument to a general scalar field. Assume the derivative 
I’ (a; y) exists for some y. Then if h ¥ 0 we can write 


hy) — 
fla + hy) — f(a) = EE ATO). 5, 
As h—~ 0 the right side tends to the limit f’(a; y)-0 = 0; hence the existence of f’(a; y) 
for a given y implies that 
lim f(a + hy) = f(a) 
h-0 


for the same y. This means that f(x) — f(a) as x — a along a straight line through a having 
the direction y. If f’(a; y) exists for every vector y, then f(x) f(a) as x — a along every 
line through a. This seems to suggest that f is continuous at a. Surprisingly enough, this 
conclusion need not be true. The next example describes a scalar field which has a direc- 
tional derivative in every direction at O but which is not continuous at O. 


EXAMPLE. Let f be the scalar field defined on R? as follows: 


CR ee ae ese ( e 
9 5 ae oe b ] 9 bd 


Let a = (0, 0) and let y = (a, b) be any vector. If a # 0 and h ¥ 0 we have 


f(a + hy) -—f(@ _ flhy) — f(O) _ flha, hb) ab" 


h h h a? + h?b* 


Letting h—0 we find f’(O; y) = b?/a. If y = (0, 5) we find, in a similar way, that 
f'(O; y) =0. Therefore f’(O; y) exists for all directions y. Also, f(x) ~0 as x > O 
along any straight line through the origin. However, at each point of the parabola x = y? 
(except at the origin) the function f has the value 3. Since such points exist arbitrarily close 
to the origin and since f(O) = 0, the function fis not continuous at O. 


The foregoing example shows that the existence of a// directional derivatives at a point 
fails to imply continuity at that point. For this reason, directional derivatives are a some- 
what unsatisfactory extension of the one-dimensional concept of derivative. A more suitable 
generalization exists which implies continuity and, at the same time, permits us to extend 
the principal theorems of one-dimensional derivative theory to the higher dimensional case. 
This is called the total derivative. 
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8.11 The total derivative 


We recall that in the one-dimensional case a function f with a derivative at a can be 
approximated near a by a linear Taylor polynomial. If f’(a) exists we let E(a, h) denote the 
difference 


fla +h) —f@) | 


(8.6) E(a, h) = ; 


f(a) if h#¥0, 


and we define E(a, 0) = 0. From (8.6) we obtain the formula 
f(ath=f/@+f'@h t+ hEG, h), 


an equation which holds also fork = 0. This is the first-order Taylor formula for approxi- 
mating f(a + h) — f(a) by f’(a)h. The error committed is hE(a, h). From (8.6) we see 
that E(a, h)—-0 as h->0Q. Therefore the error hE(a, h) is of smaller order than h for 
small h. 

This property of approximating a differentiable function by a linear function suggests a 
way of extending the concept of differentiability to the higher-dimensional case. 

Let f: S—> R be a scalar field defined on a set S in R”. Let a be an interior point of S, 
and let B(a; r) be an n-ball lying in S. Let v be a vector with ||v|| <r, so that 
a+ ve Ba;r). 


DEFINITION OF A DIFFERENTIABLE SCALAR FIELD. We say that fis differentiable at a if there 
exists a linear transformation 


T,:R"—R 
from R" to R, and a scalar function E(a, v) such that 
(8.7) f(a + v) = f(a) + T,(v) + loll EG, »), 


for ||v|| <r, where E(a, v) +0 as |\v|| +0. The linear transformation T,, is called the total 
derivative of f at a. 


Note: The total derivative 7, is a linear transformation, not a number. The function 
value T,(v) is a real number; it is defined for every point vin R". The total derivative was 
introduced by W. H. Young in 1908 and by M. Fréchet in 1911 in a more general context. 


Equation (8.7), which holds for ||v|| <r, is called a first-order Taylor formula for f(a + v). 
It gives a linear approximation, 7,,(v), to the difference f(a + v) — f(a). The error in the 
approximation is ||v|| E(a@, v), aterm which is of smaller order than ||v|| as ||v|| 0; that is, 
E(a, v) = o(|lv|]) as ||v|| +0. 

The next theorem shows that if the total derivative exists it is unique. It also tells us how 
to compute 7,(y) for every y in R”. 


The gradient of a scalar field 259 


THEOREM 8.5. Assume f is differentiable at a with total derivative T,. Then the derivative 
t'(a; y) exists for every y in R” and we have 


(8.8) T,(y) =f'(a; y). 


Moreover, f'(a; y) is alinear combination of the components of y. In fact, ify = (\1,--+sVn)s 
we have 


(8.9) filasy) = SDS (@ye- 


Proof. Equation (8.8) holds trivially if y = O since both 7,(O) = 0 and f’(a; O) = 0. 
Therefore we can assume that y ¥ O. 
Since f is differentiable at a we have a Taylor formula, 


(8.10) f(a + v) = f(a) + T,(v) + [loll E(@, ») 


for ||v|| <r for some r >0, where E(a, v) > 0 as ||v|| ~0. In this formula we take 
v = hy, where h # 0 and |A| |ly|| <r. Then |lv|| <r. Since 7, is linear we have 
T,(v) = T,(hy) = hT,(y). Therefore (8.10) gives us 


(8.11) f(a+ = -f@ _ T(y) + a+ E(a, v). 


Since || v|| 0 as h — 0 and since |A|/h = +1, the right-hand member of (8.11) tends to the 
limit 7,(y) ash > 0. Therefore the left-hand member tends to the same limit. This proves 
(8.8). 

Now we use the linearity of T, to deduce (8.9). If y=()1,...,y,) we have y = 
>" Yxlx» hence 


TA9) = To( Site) =Z yTlen) = Zyl" es) =Z DiS) 


8.12 The gradient of a scalar field 


The formula in Theorem 8.5, which expresses f’(a; y) as a linear combination of the 
components of y, can be written as a dot product, 


f(a; y) = 2 Dis (a)y, = Vf@)-y, 
where Vf(a) is the vector whose components are the partial derivatives of fat a, 


Vf(@) = (Dif(@),.--, Dr f(@)- 


This is called the gradient of f. The gradient V/is a vector field defined at each point a where 
the partial derivatives D,f(a),..., D,f(a) exist. We also write grad ffor Vf. The symbol 
V is pronounced “‘del.”’ 


260 Differential calculus of scalar and vector fields 

ha Ree oiaet Taylor formula (8.10) can now be written in the form 

(8.12) f(a + v) = f(a) + Vf(a)- v + |0|| EC, v), 

where E(a, v)—>0 as ||v|| 0. In this form it resembles the one-dimensional Taylor 


formula, with the gradient vector Vf(a) playing the role of the derivative f(a). 
From the Taylor formula we can easily prove that differentiability implies continuity. 


THEOREM 8.6. Jf a scalar field f is differentiable at a, then f is continuous at a. 
Proof. From (8.12) we have 
[f(a + v) — f(a)| = |Vf(a)- v + [loll EG, v)]. 
Applying the triangle inequality and the Cauchy-Schwarz inequality we find 
0<lfla+ ») — f@l < IVf@ loll + lel |EG@ »)I. 


This shows that f(a + v) — f(a) as ||v|| ~ 0, so fis continuous at a. 


Vv fla) 


f'(a;y) is the component 
of V/f(a) along the unit vector y 


FiIGuRE 8.5 Geometric relation of the directional derivative to the gradient vector. 


When y is a unit vector the directional derivative f’(a; y) has a simple geometric relation 
to the gradient vector. Assume that V/(a) # O and let 0 denote the angle between y and 
Vf(a). Then we have 


f' (a; y) = VA(@)- y = ||VA(@)|I lly] cos 6 = || Vf(a)|| cos 6. 


This shows that the directional derivative is simply the component of the gradient vector 
in the direction of y. Figure 8.5 shows the vectors V/f(a) and y attached to the point a. 
The derivative is largest when cos 6 = 1, that is, when y has the same direction as V/(a). 
In other words, at a given point a, the scalar field undergoes its maximum rate of change in 
the direction of the gradient vector; moreover, this maximum is equal to the length of the 
gradient vector. When V/(a) is orthogonal to y, the directional derivative f’(a; y) is 0. 


A sufficient condition for differentiability 261 


In 2-space the gradient vector is often written as 


Vf (x, y) = — Y) , n Of (x, Y) 


dy 


In 3-space the corresponding formula is 


Vie Me 2) 4 — 2) a4 ae Z) 


8.13 A sufficient condition for differentiability 


If fis differentiable at a, then all partial derivatives D,f(a),..., D,f(a) exist. However, 
the existence of all these partials does not necessarily imply that f is differentiable at a. 
A counter example is provided by the function 

2 


xy 


ve 


I(x, y) = if x40, f0,y)=0, 


x? 


discussed in Section 8.10. For this function, both partial derivatives D,f(O) and D,f(O) 
exist but fis not continuous at O, hence f cannot be differentiable at O. 

The next theorem shows that the existence of continuous partial derivatives at a point 
implies differentiability at that point. 


THEOREM 8.7. A SUFFICIENT CONDITION FOR DIFFERENTIABILITY. Assume that the partial 
derivatives D,f,..., D,f exist in some n-ball B(a) and are continuous at a. Then f is dif- 
ferentiable at a. 

Note: A scalar field satisfying the hypothesis of Theorem 8.7 is said to be continuously 
differentiable at a. 


Proof. The only candidate for 7,(v) is Vf(a)- v. We will show that 
f(a + v) — f(a) = Vf(@): v + Ilo] £@, »), 


where E(a, v) — 0 as ||v|| 0. This will prove the theorem. 

Let A = ||v||. Then v = Au, where ||u|| = 1. We keep A small enough so that a + v lies 
in the ball B(@) in which the partial derivatives D,f,..., D,fexist. Expressing u in terms 
of its components we have 


u=u,e, +°*' + 4u,2e,, 


where e,,..., @, are the unit coordinate vectors. Now we write the difference f(a + v) — 
f(a) as a telescoping sum, 


6.13) fle+v)—f@ =flat mu) —f@ =S {fe + iy) — fe+ Iy.)}, 
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where vp), 0,,..., 0, are any vectors in R” such that v) = Oand v, = u. Wechoose these 
vectors so they satisfy the recurrence relation v, = v,_, + u,e,. That is, we take 


v= O, v, = u,e,, Vo = Ue; + Ugo, ..-, VD, = Ue, +°°' + 4,2e,. 
Then the kth term of the sum in (8.13) becomes 
f(a + Avy + Aue.) — f(a + Avy_1) = f(D, + Aue) — F (by); 


where b, = a + Av,_,. ‘Vhe two points b, and b, + Au,e, differ only in their kth component. 
Therefore we can apply the mean-value theorem of differential calculus to write 


(8.14) f(b, + Auye,) — f(O,) = Au, Dif (Cx) » 
where c, lies on the line segment joining 5, to b, + Au,e,. Note that 5,—> a and hence 


c¢,—>aasi—0. 
Using (8.14) in (8.13) we obtain 


fla + 0) — f(a) = AY Dyfed 
But Vf(a):v =AVf(a)-u=AD", D, f(au;,, so 


fla +») — f(@) — Vf(@)-v = AY {Df (ey) — Dy f@}uy = loll EC@, 2), 


where 
E(@, ») = {Dif(e,) — DiS} 


Since c, — a as ||v|| — 0, and since each partial derivative D,f is continuous at a, we see 
that E(a, v) > 0 as ||v|| 0. This completes the proof. 


8.14 Exercises 


1. Find the gradient vector at each point at which it exists for the scalar fields defined by the 
following equations: 


(a) f(x, y) = x? + y* sin (xy). (d) f(x, y,z) = x? — y® + 227, 
(b) f(x, y) = e* cosy. (e) f(x, y, z) = log (x? + 2y? — 32?). 
(c) f(x, y, z) = x*y®zt. (f) f(x, y, z) = x”. 
2. Evaluate the directional derivatives of the following scalar fields for the points and directions 
given: 


(a) f(x, y, z) = x? + 2y® + 3z” at (1, 1, 0) in the direction of i — j + 2k. 
(b) f(x, y, z) = (x/y)* at (1, 1, 1) in the direction of 2i+j —k. 

3. Find the points (x, y) and the directions for which the directional derivative of f(x, y) = 
3x? + y® has its largest value, if (x, y) is restricted to be on the circle x? + y? = 1. 

4. A differentiable scalar field fhas, at the point (1, 2), directional derivatives +2 in the direction 
toward (2,2) and —2 in the direction toward (1, 1). Determine the gradient vector at (1, 2) 
and compute the directional derivative in the direction toward (4, 6). 

5. Find values of the constants a, b, and c such that the directional derivative of f(x, y, z) = 
axy* + byz + cz®x* at the point (1, 2, —1) has a maximum value of 64 in a direction parallel 
to the z-axis. 
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6. Given ascalar field differentiable at a point ain R®. Suppose that f’(a; y) = land/f’(a; z) = 2, 
where y = 2i + 3jandz =i +j. Makea sketch showing the set of all points (x, y) for which 
f'(a; xi + yj) =6. Also, calculate the gradient Vf(a). 

7. Let fand g denote scalar fields that are differentiable on an open set S. Derive the following 
properties of the gradient: 

(a) grad f = O if fis constant on S. 
(b) grad (f + g) = gradf + gradg. 
(c) grad (cf) = ¢ grad fif c is a constant. 
(d) grad (fg) = fgradg +g grad /. 


(e) grad (4) _& grad — fgradg 
g g 


8. In R® let r(x, y, z) = xi + yi + zk, and let r(x, y, z) = |lr(x, y, z)]] . 
(a) Show that Vr(x, y, z) is a unit vector in the direction of r(x, y, z). 
(b) Show that V(r”) = nr” *r if n is a positive integer. [Hint: Use Exercise 7(d).] 
(c) Is the formula of part (b) valid when zn is a negative integer or zero? 
(d) Find a scalar field fsuch that Vf =r. 
9. Assume f is differentiable at each point of an n-ball B(a). If f’(x; y) = 0 for n independent 
vectors y;,..., Y, and for every x in B(a), prove that fis constant on B(a). 
10. Assume f is differentiable at each point of an n-ball B(@). 
(a) If Vf(x) = O for every x in B(a), prove that fis constant on B(a). 
(b) If f(x) < f(a) for all x in B(a), prove that Vf(a) = O. 
11. Consider the following six statements about a scalar field f: S +R, where S © R” and a is 
an interior point of S. 
(a) fis continuous at a. 
(b) fis differentiable at a. 
(c) f’(a; y) exists for every y in R”. 
(d) All the first-order partial derivatives of f exist in a neighborhood of a and are continuous 
at a. 
(e) Vf(a) = O. 
(f) f(x) = ||x — all for all x in R”. 


at points at whichg #0. 


In a table like the one shown here, mark T 
in the appropriate square if the statement in 
row (x) always implies the statement in 
column (y). For example, if (a) always implies 
(b), mark T in the second square of the first 
row. The main diagonal has already been 
filled in for you. 


8.15 A chain rule for derivatives of scalar fields 


In one-dimensional derivative theory, the chain rule enables us to compute the derivative 
of a composite function g(t) = f[r(¢)] by the formula 


s(t) =f" ir) (2). 
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This section provides an extension of the formula when / is replaced by a scalar field 
defined on a set in n-space and r is replaced by a vector-valued function of a real variable 
with values in the domain of f. 

In a later section we further extend the formula to cover the case in which both f and r 
are vector fields. 

It is easy to conceive of examples in which the composition of a scalar field and a vector 
field might arise. For instance, suppose f(x) measures the temperature at a point x of a 
solid in 3-space, and suppose we wish to know how the temperature changes as the point x 
varies along a curve C lying in the solid. If the curve is described by a vector-valued 
function r defined on an interval [a, b], we can introduce a new function g by means of the 
formula 


g(t) =fir(t)] if ast. 


This composite function g expresses the temperature as a function of the parameter ¢, and 
its derivative g’(t) measures the rate of change of the temperature along the curve. The 
following extension of the chain rule enables us to compute the derivative g’(t) without 
determining g(t) explicitly. 


THEOREM 8.8. CHAIN RULE. Let fbe a scalar field defined on an open set S in R", and let 
r be a vector-valued function which maps an interval J from R' into S._ Define the composite 
function g = foronJ by the equation 


g(t) = f[r(t)] if ted. 


Let t be a point in J at which r'(t) exists and assume that f is differentiable at r(t). Then g'(t) 
exists and is equal to the dot product 


(8.15) g(t) = Vf(a) ‘r'(t), where a=r‘(t). 

Proof. Leta =r(t), where ¢ is a point in J at which r’(t) exists. Since S is open there ts 
an n-ball B(a) lying in S. We take Ah ¥ 0 but small enough so that r(¢ + A) lies in B(a), 
and we let y = r(t + h) — r(t). Note that y—- Oash—0. Now we have 

g(t + A) — g(t) = fire + A] — fir] = fla + y) — f@). 
Applying the first-order Taylor formula for f we have 
f(a t+ y) — f(a) = Vf(@)-y + lly EG@, y), 


where E(a, y) — 0 as ||y|| ~ 0. Since y = r(t + A) — r(f) this gives us 


g(t +h) = 2) gpg E+ MH), Mn(t + h) — (OI 


E(a, y). 
h h 5 (a, y) 


Letting h — 0 we obtain (8.15). 
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EXAMPLE |. Directional derivative along a curve. When the function r describes a curve 
C, the derivative r’ is the velocity vector (tangent to the curve) and the derivative g’ in 
Equation (8.15) is the derivative of f with respect to the velocity vector, assuming that 
r’ ~ O. If T(t) is a unit vector in the direction of r’(t) (Tis the unit tangent vector), the dot 
product Vf[r(t)]- T(t) is called the directional derivative of f along the curve C or in the 
direction of C. For a plane curve we can write 


T(t) = cos a(t)i + cos f(t) j, 


where «(f) and f(t) are the angles made by the vector 7(t) and the positive x- and y-axes; 
the directional derivative of f along C becomes 


Vf lr(t)] > T(t) = Dy flr(t)] cos a(t) + Dz flr(t)] cos B(t). 
This formula is often written more briefly as 


Os coe eee 
Ox oy 


Some authors write df/ds for the directional derivative Vf: 7. Since the directional 
derivative along C is defined in terms of 7, its value depends on the parametric representa- 
tion chosen for C. A change of the representation could reverse the direction of T; this, 
in turn, would reverse the sign of the directional derivative. 


EXAMPLE 2. Find the directional derivative of the scalar field f(x, y) = x? — 3xy along 
the parabola y = x? — x + 2 at the point (1,2). 


Solution. At an arbitrary point (x, y) the gradient vector is 


vice, y) = Li + Fj = ex — 3yyi — ay. 
Ox oy 
At the point (1,2) we have V/(1, 2) = —4i — 3j. The parabola can be represented 
parametrically by the vector equation r(¢) = ti + (t? — t + 2)j. Therefore, r(1) =i + 2j, 
r'(t)=i+ (2¢ — Lj, and r'(1) =i+ J. For this representation of C the unit tangent 
vector T(1) is GG + piv 2 and the required directional derivative is V/(1, 2): T0) = —7/V 2. 


EXAMPLE 3. Let f be a nonconstant scalar field, differentiable everywhere in the plane, 
and let c be a constant. Assume the Cartesian equation f(x, y) = c describes a curve C 
having a tangent at each of its points. Prove that f has the following properties at each 
point of C: 

(a) The gradient vector Vfis normal to C. 

(b) The directional derivative of fis zero along C. 

(c) The directional derivative of f has its largest value in a direction normal to C. 
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Solution. If Tis a unit tangent vector to C, the directional derivative of f along C is the 
dot product Vf: 7. This product is zero if Vfis perpendicular to J, and it has its largest 
value if V/is parallel to 7. Therefore both statements (b) and (c) are consequences of (a). 
To prove (a), consider any plane curve I’ with a vector equation of the form r(t) = 
X(t)i + Y(t)j and introduce the function g(t) =/[r(t)]. By the chain rule we have 
g(t) = Vf[r(t)]-r'(t). When I’ = C, the function g has the constant value c so g’(t) = 0 
if r(t)eC. Since g’ = Vf-r’, this shows that Vfis perpendicular to r’ on C; hence Vf 
is normal to C. 


8.16 Applications to geometry. Level sets. Tangent planes 


The chain rule can be used to deduce geometric properties of the gradient vector. Let f 
be a scalar field defined on a set S in R” and consider those points x in S for which f(x) has 
a constant value, say f(x) = c. Denote this set by L(c), so that 


L(c) = {x|x eS and f(x) = c}. 


The set L(c) is called a level set of f. In R?, L(c) is called a evel curve; in R? it is called a 
level surface. 


Level surface L(c) 


Lines of flow 


FIGURE 8.6 The dotted curves are iso- FiGureE 8.7 The gradient vector Vf is normal 
thermals: f(x, y) =c. The gradient vector to each curve I’ on the level surface 
Vf points in the direction of the lines of flow. [(, y, Zz) =e. 


Families of level sets occur in many physical applications. For example, if f(x, y) 
represents temperature at (x, y), the level curves of f (curves of constant temperature) are 
called isothermals. The flow of heat takes place in the direction of most rapid change in 
temperature. As was shown in Example 3 of the foregoing section, this direction is normal 
to the isothermals. Hence, in a thin flat sheet the flow of heat is along a family of curves 
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orthogonal to the isothermals. These are called the /ines of flow; they are the orthogonal 
trajectories of the isothermals. Examples are shown in Figure 8.6. 

Now consider a scalar field f differentiable on an open set S in R*, and examine one of its 
level surfaces, L(c). Let a be a point on this surface, and consider a curve I’ which lies on 
the surface and passes through a, as suggested by Figure 8.7. We shall prove that the 
gradient vector V/f(a) is normal to this curve at a. That is, we shall prove that V/(a) is 
perpendicular to the tangent vector of [ at a. For this purpose we assume that I is 


Vf, a normal vector 


Tangent plane 


vg, 


ee ~~ Level surface L(c) 


eb 


FiGuRE 8.8 The gradient vector Vf is normal to the tangent plane of a level surface 
[(%, y, 2) =e. 


described parametrically by a differentiable vector-valued function r defined on some interval 
Jin R*. Since I lies on the level surface L(c), the function r satisfies the equation 


Sf [r(t)] = c for all tin J. 
If g(t) = f[r(t)] for t in J, the chain rule states that 
g(t) = Vir] re). 
Since g is constant on J, we have g’(t) = 0 on J. In particular, choosing f, so that g(t,) = a 
we find that 


Vf(a)-r'(ty) = 0. 


In other words, the gradient of f at a is perpendicular to the tangent vector r’(t,), as 
asserted. 
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Now we take a family of curves on the level surface L(c), all passing through the point 
a. According to the foregoing discussion, the tangent vectors of all these curves are per- 
pendicular to the gradient vector Vf(a). If V/(a@) is not the zero vector, these tangent vectors 
determine a plane, and the gradient V/f(a) is normal to this plane. (See Figure 8.8.). This 
particular plane is called the tangent plane of the surface L(c) at a. 

We know from Volume I that a plane through a with normal vector N consists of all 
points x in R® satisfying N- (x — a) = 0. Therefore the tangent plane to the level surface 
L(c) at a consists of all x in R® satisfying 


Vf(a):- (x — a) = 0. 


To obtain a Cartesian equation for this plane we express x, a, and Vf(a) in terms of their 
components. Writing x = (x,y,z), @ = (x,y, 2,), and 


Vf (a) = Di f(@it+ Dof(@j + Dsf(ak, 


we obtain the Cartesian equation 


D, f(a)(x — xX) + D, f (ayy — yy) + D3; f(a)(z —z,)=0. 


A similar discussion applies to scalar fields defined in R?. In Example 3 of the foregoing 
section we proved that the gradient vector V/(a) at a point a of a level curve is perpendicular 
to the tangent vector of the curve at a. Therefore the tangent line of the level curve L(c) at 
a point a = (x, y,) has the Cartesian equation 


D, f(a)(x — x1) + Do f(ay — yi) = 0. 


8.17. Exercises 


1. In this exercise you may assume the existence and continuity of all derivatives under con- 
sideration. The equations u = f(x, y), x = X(t), y = Y(t) define uw as a function of tf, say 
u= F(t). 

(a) Use the chain rule to show that 
a ay 


F'(t) = 5) + a Y(t), 


where 0f/0x and f/@y are to be evaluated at [X(t), Y(t)]. 
(b) In a similar way, express the second derivative F "(t) in terms of derivatives of f, X, and 
Y. Remember that the partial derivatives in the formula of part (a) are composite functions 
given by 
af af 
ax 


= Df{xX(t), YO), = D,f{X(t), Y(0)). 


ay = 
2. Refer to Exercise | and compute F’(t) and F”(t) in terms of t for each of the following special 
cases: 
(a) fy) =xet+y?, X() =t, Yu) =t?*. 
(b) f(x, y) = e™ cos (xy*), X(t) =cost, Y(t) = sin. 
(c) f(x, y) =log [U1 + eV + e”)], X(t) =e, Y(t)' =e. 


12. 
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. In each case, evaluate the directional derivative of f for the points and directions specified: 


(a) f(x, y, z) = 3x — Sy + 2zat (2, 2, 1) in the direction of the outward normal to the sphere 
xe+ yr +2722 =9, 

(b) f(x, y, z) = x? — y? at a general point of the surface x? + y? + z? = 4 in the direction 
of the outward normal at that point. 

(c) f(x, y,z) =x? + y? — z? at (3, 4, 5) along the curve of intersection of the two surfaces 
2x* + 2y? — z? = 25 and x? + y® = z?, 


. (a) Find a vector V(x, y, z) normal to the surface 


z= Jx? + y2 + (x? + y?)28 


at a general point (x, y, z) of the surface, (x, y, z) # (0, 0,0). 
(b) Find the cosine of the angle 6 between V(x, y, z) and the z-axis and determine the limit of 
cos 8 as (x, y, z) — (0, 0, 0). 


. The two equations e“ cos v = x and e” sinv = y define u and v as functions of x and y, say 


u = U(x, y)andv = V(x, y). Find explicit formulas for U(x, y) and V(x, y), valid for x > 0, 
and show that the gradient vectors VU(x, y) and VV (x, y) are perpendicular at each point 
(x, y). 


. Let f(x, y) = VJ/|xyl- 


(a) Verify that 0f/@x and 0f/0y are both zero at the origin. 
(b) Does the surface z = f(x, y) have a tangent plane at the origin? [Hint: Consider the 
section of the surface made by the plane x = y.] 


. If (Xo, Yo, Zo) iS a point on the surface z = xy, then the two lines z = yox, y = ygandz = xgy, 


X =X, intersect at (x9, Yo, Zo) and lie on the surface. Verify that the tangent plane to this 
surface at the point (x9, yo, Zo) contains these two lines. 


. Find a Cartesian equation for the tangent plane to the surface xyz = a® at a general point 


(Xo, Yo. Zo). Show that the volume of the tetrahedron bounded by this plane and the three 
coordinate plane is 9a°/2. 


. Find a pair of linear Cartesian equations for the line which is tangent to both the surfaces 


x? + y? + 227 = 4andz = e*” at the point (1, 1, 1). 


. Find a constant c such that at any point of intersection of the two spheres 


(x-c?+y4+2=3 and x 4+(y—-1%?4+2%= 


the corresponding tangent planes will be perpendicular to each other. 


. If r,; and r, denote the distances from a point (x, y) on an ellipse to its foci, show that the 


equation r,; + ’, = constant (satisfied by these distances) implies the relation 
T- V(y + rp) = 0, 


where T is the unit tangent to the curve. Interpret this result geometrically, thereby showing 
that the tangent makes equal angles with the lines joining (x, y) to the foci. 

If Vf(x, y, z) is always parallel to xi + yj + zk, show that f must assume equal values at the 
points (0, 0, a) and (0, 0, —a). 


8.18 Derivatives of vector fields 


Derivative theory for vector fields is a straightforward extension of that for scalar fields. 


Let f: S > R™ be a vector field defined on a subset S of R". Ifa is an interior point of S 


270 Differential calculus of scalar and vector fields 
and if y is any vector in R” we define the derivative f’(a; y) by the formula 


fla + hy) — f(@) 


f'(a; y) = lim 
h>0 h 


whenever the limit exists. The derivative f’(a; y) is a vector in R”. 
Let f, denote the kth component of f. We note that the derivative f’(a; y) exists if and 
only if f, (a; y) exists for each k = 1, 2,..., m, in which case we have 


fas) = Pilar y),..., fiasy)) => fila Veg, 


where e, is the kth unit coordinate vector. 
We say that fis differentiable at an interior point a if there is a linear transformation 


T,:R" — R”™ 
such that 


(8.16) fla + v) = f(a) + T(r) + ||| EG, »), 
where E(a, v) > O as v-> O. The first-order Taylor formula (8.16) is to hotd for all v 


with ||vl|| <r for some r>0. The term E(a, v) is a vector in R™. The linear trans- 
formation T, is called the total derivative of fat a. 
For scalar fields we proved that 7,(y) is the dot product of the gradient vector V/(a) 


with y. For vector fields we will prove that T,(y) is a vector whose kth component is the dot 
product V/,(a)-y. 


THEOREM 8.9. Assume f is differentiable at a with total derivative T,. Then the derivative 
f' (a; y) exists for every a in R", and we have 


(8.17) Ty) =f'(a; y). 


Moreover, if f = (fi ,.-- 5fm) and ify = (y1,--+- Vn), we have 


(6.18) T(9) => Vila) “yee = (Vfila) -y,«.« Vhnla) 9), 


Proof. We argue as in the scalar case. If y = O, then f’(a; y) = Oand 7T,(O) = O. 
Therefore we can assume that y # O. Taking v = hy in the Taylor formula (8.16) we have 


f(a + hy) — f(a) = T,(hy) + \|hy|| E(a, v) = hT,(y) + Al \ly|| E@, v). 


Dividing by / and letting : — 0 we obtain (8.17). 
To prove (8.18) we simply note that 


f(a; y) => fi (@s Y) &, => VfA@) * Y ey. 


Differentiability implies continuity 27] 
Equation (8.18) can also be written more simply as a matrix product, 


T,(y) = Df(a)y, 


where Df(a) is the m x n matrix whose kth row is Vf,(a), and where y ts regarded as an 
n X | column matrix. The matrix Df(a) is called the Jacobian matrix of fata. Its kj entry 
is the partial derivative D, f,(a). Thus, we have 


| Difila) Defi(a) -*: D,fi(@) 
D, f(a) Defra) ++: D, f(a) 
Df(a) = 


| Dy fn (@) Dz fn(@) eer Di Sm(a)_ 


The Jacobian matrix Df(a) is defined at each point where the mn partial derivatives 
D,f,(a) exist. 

The total derivative 7, is also written as f’(a). The derivative f’(a) is a linear trans- 
formation; the Jacobian Df(a) is a matrix representation for this transformation. 

The first-order Taylor formula takes the form 


(8.19) fat v) =f(@) +f @(v) + loll E@, »), 
where E(a, v) > O as v-> O. This resembles the one-dimensional Taylor formula. To 


compute the components of the vector f’(a)(v) we can use the matrix product Df(a)v or 
formula (8.18) of Theorem 8.9. 


8.19 Differentiability implies continuity 


THEOREM 8.10. Jfa vector field f is differentiable at a, then f is continuous at a. 


Proof. As in the scalar case, we use the Taylor formula to prove this theorem. If we let 
v— O in (8.19) the error term ||v|| E(a, v) ~ O. The linear part f’(a)(v) also tends to O 
because linear transformations are continuous at O. This completes the proof. 


At this point it is convenient to derive an inequality which will be used in the proof of 
the chain rule in the next section. The inequality concerns a vector field f differentiable at 
a; it states that 


(8.20) If'@()|| <_M,(a) lv], where M,(a) = SIVA@. 


To prove this we use Equation (8.18) along with the triangle inequality and the Cauchy- 
Schwarz inequality to obtain 


If'(a(v)|| = 


SVA@ vee | SIV@)- ol] SY IVAC@)l Nol. 
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8.20 The chain rule for derivatives of vector fields 


THEOREM 8.11. CHAIN RULE. Let f and g be vector fields such that the composition 
h = fo g is defined in a neighborhood of a point a. Assume that g is differentiable at a, with 
total derivative g'(a). Let b = g(a) and assume that f is differentiable at b, with total 
derivative f'(b). Then h is differentiable at a, and the total derivative h(a) is given by 


h(a) =f'(b)°8'(@), 
the composition of the linear transformations f'(b) and g’(a). 


Proof. Weconsider the difference h(a + y) — h(a) for small ||y||, and show that we have 
a first-order Taylor formula. From the definition of 4 we have 


(8.21) h(a + y) — h(a) = f[g(a + y)] — flg(a)] = f(6 + ») — f(b), 
where v = g(a + y) — g(a). The Taylor formula for g(a + y) gives us 

(8.22) v=g'(a(y)+ lyll E,(a,y), where E,(a,y)>O as yO. 
The Taylor formula for f(b + v) gives us 

(8.23) f(b + v) — f(b) =f '(b)(») + lvl] EG, »), 

where E,(b, v) > O as v-> O. Using (8.22) in (8.23) we obtain 


(8.24) f(b+ v) — f(b) =f' (dg (ay) +f (yl £,(@, y)) + loll EC, v) 
= f'(b)g’(a)(y) + |ly|| Ela, y), 
where E(a, O) = O and 


(8.25) E(a, y) = f'(6\E,(a, y)) + = E(b,v) if yO. 


To complete the proof we need to show that E(a, y) > Oasy— O. 

The first term on the right of (8.25) tends to O as y > O because E,(a, y) > Oasy > O 
and linear transformations are continuous at O. 

In the second term on the right of (8.25) the factor E,(b, v) > O because v > O as 
y—O. The quotient ||v||/||y|| remains bounded because, by (8.22) and (8.20) we have 


lol] < M,(@) lly] + Iyll TE, y) I. 


Therefore both terms on the right of (8.25) tend to O as y > O, so E(a, y) > O. 
Thus, from (8.24) and (8.21) we obtain the Taylor formula 


h(a + y) — h(a) = f'(b)g"(a)(y) + llyll E(@, y), 
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where E(a, y)—> O as y-> O. This proves that h is differentiable at a and that the total 
derivative A’(a) is equal to the composition f ‘(d) ° g’(a). 


8.21 Matrix form of the chain rule 


Let h = fog, where g is differentiable at a and fis differentiable at b = g(a). The chain 
rule states that 


h'(a) = f'(b) g(a). 
We can express the chain rule in terms of the Jacobian matrices Dh(a), Df(b), and Dg(a) 
which represent the linear transformations h’(a), f(b), and g’(a), respectively. Since 
composition of linear transformations corresponds to multiplication of their matrices, we 
obtain 
(8.26) Dh(a) = Df(b) Dg(a), where b = g(a). 
This is called the matrix form of the chain rule. It can also be written as a set of scalar 


equations by expressing each matrix in terms of its entries. 
Suppose that ae R?, b = g(a) ER", and f(b) ER”. Then A(a) € R™ and we can write 


& = (£1,---58n)> fH is xatas h= (hy,...,hp)- 


Then Dh(a) is anm x p matrix, Df(b) isanm x n matrix, and Dg(a) is ann X p matrix, 
given by 


Dh(a) = [D;h(@]; 321, Df(b) = [D,f(B)ineir, — Dg(a@) = (D2, (a) he 3-1 - 


The matrix equation (8.26) is equivalent to mp scalar equations, 
D h(a) = > D,f(b)D;2,{a) , for i=1,2,...,m and j=1,2,...,p. 
k=1 


These equations express the partial derivatives of the components of # in terms of the 
partial derivatives of the components of fand g. 


EXAMPLE |. Extended chain rule for scalar fields. Suppose f is a scalar field (m= 1). 
Then / is also a scalar field and there are p equations in the chain rule, one for each of the 
partial derivatives of h: 


D ,;h(a) = 2 Di f(B)D s8:(4): for jp S12, 23350: 


The special case p = 1 was already considered in Section 8.15. In this case we get only one 
equation, 


h'(@) => D,S(B)ei(a). 
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Now take p = 2 andn=2. Write a = (s, t) and b = (x, y). Then the components x 
and y are related to s and ¢ by the equations 


x= g\(5,t), y= ,(s,¢). 


The chain rule gives a pair of equations for the partial derivatives of h: 


D,h(s, t) a Dif (x, y) Digi(s, t) + Def (x; y) D,g2(s, t), 
D.A(s, t) = Dif (x, y) Dog\(5, t) T Dof (x; y) D2g2(5, t). 


In the d-notation, this pair of equations is usually written as 


(8.27) Oh _ Of ox | Ofdy ch _ ofdx | of ay 
) ds axa@s Oy Os Ot Oxat Oy ot 


EXAMPLE 2. Polar coordinates. The temperature of a thin plate is described by a scalar 
field f, the temperature at (x, y) being f(x, y). Polar coordinates x = rcos#, y =rsin 0 
are introduced, and the temperature becomes a function of r and 6 determined by the 
equation 

g(r, 0) = f(r cos 0, r sin 8). 


Express the partial derivatives 0gy/dr and 0/06 in terms of the partial derivatives Of/dx 
and df/dy. 


Solution. We use the chain rule as expressed in Equation (8.27), writing (r, 0) instead of 
(s,¢), and @ instead of h. The equations 


x=rcos#é, y=rsin0 


give us 
Les DY Sao Le DY baad: 
r Or 00 
Substituting these formulas in (8.27) we obtain 
0 0 0 0 0 7] 
(8.28) aes Ter arte Betis Ogee ta ga 
Or Ox Oy 06 Ox oy 


These are the required formulas for dg/dr and dq/00. 


EXAMPLE 3. Second-order partial derivatives. Refer to Example 2 and express the second- 
order partial derivative 0?q/00? in terms of partial derivatives of /. 


Solution. We begin with the formula for dq/@0 in (8.28) and differentiate with respect 
to 0, treating r as a constant. There are two terms on the right, each of which must be 
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differentiated as a product. Thus we have 


3 
a¢ = _» F Asin 8) ?) — rsin 0 ot r of A(cos 8) ’) + rcos 6— Abe, 
06 \dx Oy 06 00 \dy 


= —reos 950 — rsind (2) — rsino + reos = (5). 
0 00\ex 0 00 \dey 


To compute the derivatives of df/dx and df/dy with respect to 6 we must keep in mind that, 
as functions of r and 6, df/0x and Of/dy are composite functions given by 


og f(r cos 6, r sin 6) and ue of(r cos 6, r sin 8). 


Ox Oy 


Therefore, their derivatives with respect to 0 must be determined by use of the chain rule. 
We again use (8.27), with f replaced by D,/f, to obtain 


2) _ WDNe , AON ay _ Pek 
00 Ox 


Ox Ox 06 dy 00 ax? 
Similarly, using (8.27) with f replaced by Df, we find 


Of\ _ADef)ax AD fay Ff af 
(5) ~ a. ip ay 00 ax Oy | a er a 


When these formulas are used in (8.29) we obtain 


QO? 0 3 0? Q” 
ORT LITRE Ct ER CATT u 
06? Ox ox? Oy Ox 
0 0” 2 
Be eee ee i ee 
oy Ox Oy oy? 


This is the required formula for d?g/06?. Analogous formulas for the second-order partial 
derivatives 0q/dr*, 0%@/(Or 060), and d%q/(06 Or) are requested in Exercise 5 of the next 
section. 


8.22 Exercises 


In these exercises you may assume differentiability of all functions under consideration. 
1. The substitution ¢ = g(x, y) converts F(t) into f(x, y), where f(x, y) = Flg(x, y)]. 


(a) Show that 
of of 
ae oe 


og og 
= F [g(x, Whee and = F [g(x, ars 


(b) Consider the special case F(t) = e!™*, 9(x, y) = cos (x7 + y*). Compute 4f/ax and 
df/%y by use of the formulas in part (a). To check your result, determine f(x, y) explicitly in 
terms of x and y and compute df/0x and 4f/ ay directly from f 
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. The substitution u = (x — y)/2,v = (x + y)/2changes f(u, v) into F(x, y). Use an appropriate 


form of the chain rule to express the partial derivatives 0F/0x and 0F/ dy in terms of the partial 
derivatives 0f/éu and df] dv. 


. The equations u = f(x, y), x = X(s,t), y = Y(s,t) define u as a function of s and f¢, say 


u = F(s,t). 

(a) Use an appropriate form of the chain rule to express the partial derivatives 0F/@s and 
OF /ét in terms of df/dx, af/ay, AaX/as, AX/dt, AY/as, AY] At. 

(b) If af/(ax dy) = a2f/(@y ax), show that 


ar af ax ete) axoY @f af aY A 


as? ax as? | ax2\ as ‘as as ax dy dy as? dy*\ ds 


(c) Find similar formulas for the partial derivatives 0?F/(@s or) and 0?F/ at?. 


. Solve Exercise 3 in each of the following special cases: 


(a) X(s,t) =s +t, Y(s,t) = st. 
(b) X(s,t) = st, Y(s,t) = s/t. 
(c) X(s,t) = (s —1¢)/2, Y(s,t) =(s + 1£)/2. 


. The introduction of polar coordinates changes f(x, y) into g(r, 6), where x = rcos 6 and 


y =rsin 6. Express the second-order partial derivatives 0" q/dr?, @2y/(@r 00), and 0 q/(20 ar) 
in terms of the partial derivatives of f- You may use the formulas derived in Example 2 of 
Section 8.21. 


. The equations u = f(x, y,z), x =X(r,s,t), y = Y(r,s,t), and z = Z(r, s, t) define u as a 


function of r,s, and tf, say u = F(r, s,t). Use an appropriate form of the chain rule to express 
the partial derivatives 0F/ or, 0F/ 9s, and OF/@t in terms of partial derivatives of f, X, Y, and Z. 


. Solve Exercise 6 in each of the following special cases: 


(a) X(r,s,t) =rt+s4+t, Yr,s,t) =r—2s4+3t, Z(r,s,t)=2r+s5—-t. 
(b) X¢,s5) =P +2427, VYe,st)er—-st—t?, Zer,s,th=r—st +t?. 


. The equations u = f(x, y,z), x = X(s,t), y = Y(s,t), z = Z(s,¢) define u as a function of 


sand t, say u = F(s,t). Use an appropriate form of the chain rule to express the partial de- 
rivatives 0F/0s and 0F/0dt in terms of partial derivatives of f, X, Y, and Z. 


. Solve Exercise 8 in each of the following special cases: 


(a) X(s,t) =s? + 12?, Y(s,t)=s?—t?, Z(s,t) =2st. 

(b) X(s,t) =s +f, Y(s,t)=s —-t, Z(s,t) = St. 

The equations u = f(x, y), x = X(r,5,t), y = Y(r, 5, t) define u as a function of r, s, and f, 
say u = F(r,s,t). Use an appropriate form of the chain rule to express the partial derivatives 
0F/ er, 0F/0s, and OF/0t in terms of partial derivatives of f, X, and Y. 

Solve Exercise 10 in each of the following special cases: 

(a) X(r,s,t)h=rts, Y(r,s,t) =t. 

(b) X(r,s,t) =r+ts +t, Y(ir,sth=r?+s7 +07, 

(c) X(r,s,t) =r/s, Y(r,s,t) =s/t. 


. Let h(x) = f[g(x)], where g = (g,,...,2n) is a vector field differentiable at a, and f is a 


scalar field differentiable at 6b = g(a). Use the chain rule to show that the gradient of 4 can 
be expressed as a linear combination of the gradient vectors of the components of g, as 
follows: 


Vh(a) = >) D,, f(b) Vg;,(a) . 
k=1 


. (a) If f(x, y,z) = xi + yj + zk, prove that the Jacobian matrix Df(x, y, z) is the identity 


matrix of order 3. 
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(b) Find all differentiable vector fields f: R? — R? for which the Jacobian matrix Df(x, y, z) 

is the identity matrix of order 3. 

(c) Find all differentiable vector fields f: R? — R® for which the Jacobian matrix is a diagonal 

matrix of the form diag (p(x), g(y), r(z)), where p, g, and r are given continuous functions. 
14. Let f: R? — R2 and g: R® — R? be two vector fields defined as follows: 


f(x, y) = e**?4%% + sin (y + 2x) J, 
g(u,v,w) = (u + 2v? + 3w3)i + (20 — u)j. 
(a) Compute each of the Jacobian matrices Df(x, y) and Dg(u, v, w). 
(b) Compute the composition A(u, v, w) = f[g(u, v, w)). 
(c) Compute the Jacobian matrix DA(1, —1, 1). 
15. Let f: R? — R2 and g: R®? — R? be two vector fields defined as follows: 
fx, yz2.=(P@+y+z2zi+Qx+y4+2*)f, 
g(u,v, w) = uv?wt + w?sinoyj + urerk. 
(a) Compute each of the Jacobian matrices Df(x, y, z) and Dg(u, v, w). 


(b) Compute the composition A(u, v, w) = f[g(u, v, w)]. 
(c) Compute the Jacobian matrix Dh(u, 0, w). 


*8.23 Sufficient conditions for the equality of mixed partial derivatives 


If fis a real-valued function of two variables, the two mixed partial derivatives D,.f 
and D,,/f are not necessarily equal. By D,.f we mean D,(D,f) = 02//(0x dy), and by 
D,,f we mean D,(D,f) = 0?f/(dy dx). For example, if fis defined by the equations 
2 


x x 


x? + y? 


F(x, y) = xy for (x,y) #(0,0), f(0,0)=0, 


it is easy to prove that D, , f(0, 0) = —land D,.f(0, 0) = 1. This may be seen as follows: 
The definition of D, , (0, 0) states that 


(8.30) D,.,f(0, 0) = lim a a 
k-0 


Now we have 


D,f(0, 0) = tim LO FO.9) © 


h-0 h 


0 
and, if (x, y) ¥ (0, 0), we find 
y(x* + 4x?y? — y’*) 
D XxX, — = — a. = ee fe Ye e 
if ( y) (x? ee yy 


Therefore, if k 4 0 we have D, f(0, k) = —k*/k* = —k and hence 


D,f(0, k) — Df(0, 0) _ 
k 


—1. 
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Using this in (8.30) we find that D,,f(0,0) = —1. A similar argument shows that 
D, 2f(0, 0) = 1, and hence D,, f(0, 0) ¥ D, 2 f(0, 0). 

In the example just treated the two mixed partials D,.f and D,,f are not both con- 
tinuous at the origin. It can be shown that the two mixed partials are equal at a point 
(a, b) if at least one of them is continuous in a neighborhood of the point. We shall prove 
first that they are equal if both are continuous. More precisely, we have the following 
theorem. 


THEOREM 8.12. A SUFFICIENT CONDITION FOR EQUALITY OF MIXED PARTIAL DERIVATIVES. 
Assume f is a scalar field such that the partial derivatives D,f, D.f, D,,.f, and Dz, f exist on 
an open set S. If (a, b) is a point in S at which both D,,f and D,,f are continuous, we have 


(8.31) Dy of(a, b) = D1 f(a, b). 


Proof. Choose nonzero h and k such that the rectangle R(A, k) with vertices (a, b), 
(a+h,b),(a+h,b +k), and (a, b + k) lies in S. (An example is shown in Figure 8.9.) 


(a, b+ k) (at+h, b+ k) 


(a, 5) (a +h, b) 


FiGurE 8.9 A(h, k) is a combination of values of f at the vertices. 
Consider the expression 
Ath, k) =f(a+h,b+k)—f(at+h, b) —f(a,b +k) + f(a, b). 
This is a combination of the values of fat the vertices of R(h, k), taken with the algebraic 
signs indicated in Figure 8.9. We shall express A(h, k) in terms of D,, f and also in terms 


of D, of. 
We consider a new function G of one variable defined by the equation 


G(x) = f(x, 6 + k) — f(x, 5) 
for all x between a anda + h. (Geometrically, we are considering the values of f at those 


points at which an arbitrary vertical line cuts the horizontal edges of R(h, k).) Then we 
have 


(8.32) A(A, k) = G(a + h) — G(a). 
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Applying the one-dimensional mean-value theorem to the right-hand member of (8.32) we 
obtain G(a + h) — G(a) = hG'(x,), where x, lies between a anda+h. Since G’(x) = 
D, f(x, 6 + k) — Dyf(x, 6), Equation (8.32) becomes 


(8.33) A(h, k) = A[D, f(x, 5 + k) — Di f(%, 5)). 
Applying the mean-value theorem to the right-hand member of (8.33) we obtain 
(8.34) Ath, k) = hkDz1f (1, 1); 


where y, lies between b and b +k. The point (x, y,) lies somewhere in the rectangle 
Rh, k). 

Applying the same procedure to the function H(y) = f(a + h, y) — f(a, y) we find a 
second expression for A(h, k), namely, 


(8.35) ACh, k) = hKD, of (x2; a) ; 


where (x2, y2) also lies in R(h, k). Equating the two expressions for A(h, k) and cancelling 
hk we obtain 


Dy of (X15 V1) = Dorf (Xe. Yo) - 
Now we let (h, k) — (0, 0) and use the continuity of D,.fand D,,fto obtain (8.31). 


The foregoing argument can be modified to prove a stronger version of Theorem 8.12. 


THEOREM 8.13. Let f be a scalar field such that the partial derivatives D,f, D,f, and D,,f 
exist on an open set S containing (a, b). Assume further that D,,f is continuous on S. Then 
the derivative D,,.f(a, b) exists and we have 


Dy f(a, 6) = Dei f(a, 5). 


Proof. We define A(h, k) as in the proof of Theorem 8.12. The part of the proof leading 
to Equation (8.34) is still valid, giving us 


ACh, k) 


(8.36) = 


= Daif (% » V1) 


for some (x, y,) in the rectangle R(h, k). The rest of the proof is not applicable since it 
requires the existence of the derivative D, . f(a, b), which we now wish to prove. 
The definition of D, ,f(a, b) states that 


(8.37) D, of (a, b) = lim Dpf(a + h, b) — Dof(a, b) 


h-0 h 
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We are to prove that this limit exists and has the value D, , f(a, 5). From the definition of 
D, f we have 


Dia Sime 2 


k-0 k 
and 


fa +h, b+k)—fla +h, b) 


D, f(a + h, b) = lim 
k70 k 


Therefore the difference quotient in (8.37) can be written as 
D, f(a tr h, b) ia D, f(a, b) = lim ACh, k) 
h ko hk 
Using (8.36) we can rewrite this as 


D.f(a + h, b) — D2f(a, b) 


(8.38) : 


= lim D2 f(*1, y1). 
k-0 
To complete the proof we must show that 


(8.39) lim tim Deaf Cer, | = Dz, f(a, b). 
k 


h-0 0 


When k — 0, the point y, > b, but the behavior of x, as a function of k is unknown. 
If we knew that x, approached some limit, say ¥, as k > 0, we could use the continuity of 
D,,f to deduce that 


om Do if (X15 V1) = Dei f(x, 6). 


Since the limit would have to lie in the interval a << ¥ < a +h, we could then let h > 0 
and deduce (8.39). However, the fact that ¥ depends on k in an unknown fashion makes a 
slightly more involved argument necessary. 

Because of Equation (8.38) we know that the following limit exists: 


lim Doi f(%1, yi). 
k-0 
Let us denote this limit by F(h). To complete the proof we must show that 


lim F(h) = D,,,f(a, b). 


For this purpose we appeal to the definition of continuity of D,,f at (a, 5). 
Let « be a given positive number. Continuity of D,, f at (a, b) means that there is an 
open disk N with center (a, b) and radius 0, say, such that 


(8.40) IDoif (x,y) — Deaf (a, b)| — whenever (x, y)EN. 
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If we choose A and k so that |h| < 6/2 and |k| < 6/2, the entire rectangle shown in Figure 
8.9 will lie in the neighborhood N and, specifically, the point (x, y,) will bein N. Therefore 
(8.40) is valid when (x, y) = (%;, y,) and we can write 


(8.41) 0 < [Dea f(%s Ys) — Daa f(@, BY <5. 


Now keep A fixed and let k >0. The term D,,f(x1, y:) approaches F(h) and the other 
terms in (8.41) are independent of k. Therefore we have 


0 <IF(h) — Dafa BIS 5 <e, 


provided that 0 < |h| < 6/2. But this is precisely the meaning of the statement 


lim F(h) = Dp, f(a, b) 
h-0 


and, as we have already remarked, this completes the proof. 


Note: It should be observed that the theorem is also valid if the roles of the two 
derivatives D,.fand D,,, fare interchanged. 


8.24 Miscellaneous exercises 


1. 


Find a scalar field f satisfying both the following conditions: 

(a) The partial derivatives D,/f(0, 0) and D, f(0, 0) exist and are zero. 

(b) The directional derivative at the origin in the direction of the vector i + j exists and has 
the value 3. Explain why such an f cannot be differentiable at (0, 0). 


. Let f be defined as follows: 


xe —y 
f(y) =) ey if (x,y) #(0,0), f(0,0) =0. 
Compute the following partial derivatives, when they exist: D,f(0, 0), Do f(0, 0), Des f(O, 9), 
D, 2f (0, 9). 


3 


ay 


. Let f(x, y) = = if (, y) ¥ (0, 0), and define f(0, 0) = 0. 


x* + y8 
(a) Prove that the derivative f’(O; a) exists for every vector a and compute its value in terms 
of the components of a. 
(b) Determine whether or not fis continuous at the origin. 


. Define f(x, y) = [Vere dt forx >0,y >0. Compute 0f/0x in terms of x and y. 
. Assume that the equations u = f(x, y), x = X(t), y = Y(t) define u as a function of ¢, say 


u = F(t). Compute the third derivative F(t) in terms of derivatives of f, X, and Y. 


. The change of variables x =u +0, y = uv* transforms f(x, y) into g(u, v). Compute the 


value of @g/(dv du) at the point at which u = 1, v = 1, given that 


ay Ax® ay?” ax By By ax 
at that point. 
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7. 


10. 


14, 


The change of variables x = uv, y = 3(u® — v?) transforms f(x, y) to g(y, v). 

(a) Calculate dg/0u, og/dv, and 0%g/(du av) in terms of partial derivatives of f. (You may 
assume equality of mixed partials.) 

(b) If || Vf(x, y)||? = 2 for all x and y, determine constants a and b such that 


. Two functions F and G of one variable and a function z of two variables are related by the 


equation 
[F(x) + Gy) Pe = 2F'(x)G'(y) 


whenever F(x) + G(y) #0. Show that the mixed partial derivative D, ,z(x, y) is never zero. 
(You may assume the existence and continuity of all derivatives encountered.) 


. Ascalar field fis bounded and continuous on a rectangle R = [a, b] x [c,d]. A new scalar 


field ¢ is defined on R as follows: 


gu, v) = PL [fey dx | dy. 


(a) It can be shown that for each fixed u in [a, b] the function A defined on [c, d] by the 
equation A(y) = {% f(x, y) dx is continuous on [c, d]. Use this fact to prove that @g/@v exists 
and is continuous on the open rectangle S = (a, b) x (c, d) (the interior of R). 

(b) Assume that 


[fresnel] a fel freanay] a 


for all (u,v) in R. Prove that g is differentiable on S and that the mixed partial derivatives 
D, 2g(u, v) and Dz ,g(u, v) exist and are equal to f(u, v) at each point of S. 

Refer to Exercise 9. Suppose u and v are expressed parametrically as follows: u = A(t), 
v = Bt); and let y(t) = g[A(t), B(t)). 

(a) Determine ¢’(t) in terms of f, A’, and B’. 

(b) Compute ¢’(t) in terms of ¢ when f(x, y) = e**” and A(t) = B(t) = t?. (Assume R lies 
in the first quadrant.) 


. If f(x, y, Zz) = (r x A): (r X B), where r = xi + yj + zk and A and B are constant vectors, 


show that Vf(x, y,z) = B x (r x A) +A X (r X B). 


. Letr = xi + yj + zk and let r = |[r||. If A and Bare constant vectors, show that: 


r 


l 3A:rB-r A-B 
(b) B-V A:'VIi- a Ta a 3 . 
r r r 


] A:r 
@) 4-v(-) = - 3° 


. Find the set of all points (a, 5, c) in 3-space for which the two spheres (x — a)? + (y — 6)? + 


(z —c)? = 1 and x? + y® +z? = 1 intersect orthogonally. (Their tangent planes should be 
perpendicular at each point of intersection.) 

A cylinder whose equation is y = f(x) is tangent to the surface z? + 2xz + y =0 at all 
points common to the two surfaces. Find f(x). 
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APPLICATIONS OF DIFFERENTIAL CALCULUS 


9.1 Partial differential equations 


The theorems of differential calculus developed in Chapter 8 have a wide variety of 
applications. This chapter illustrates their use in some examples related to partial differen- 
tial equations, implicit functions, and extremum problems. We begin with some elemen- 
tary remarks concerning partial differential equations. 

An equation involving a scalar field fand its partial derivatives is called a partial differen- 
tial equation. Two simple examples in which fis a function of two variables are the first- 
order equation 
9.1) f(x, Y) _ 4 

Ox 


and the second-order equation 


(9.2) oe) Ee F(x Y) =. 

ox* Oy" 

Each of these is a homogeneous J/inear partial differential equation. That is, each has the 
form L(f) = 0, where L is a linear differential operator involving one or more partial 
derivatives. Equation (9.2) is called the two-dimensional Laplace equation. 

Some of the theory of linear ordinary differential equations can be extended to partial 
differential equations. For example, it is easy to verify that for each of Equations (9.1) 
and (9.2) the set of solutions is a linear space. However, there is an important difference 
between ordinary and partial linear differential equations that should be realized at the 
outset. We illustrate this difference by comparing the partial differential equation (9.1) 
with the ordinary differential equation 


(9.3) cog ale 


The most general function satisfying (9.3) is f(x) = C, where C is an arbitrary constant. 
In other words, the solution-space of (9.3) is one-dimensional. But the most general 
function satisfying (9.1) is 

F(x, y) = By), 
283 
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where g is any function of y. Since g is arbitrary we can easily obtain an infinite set of 
independent solutions. For example, we can take g(y) = e™ and let c vary over all real 
numbers. Thus, the solution-space of (9.1) is infinite-dimensional. 

In some respects this example is typical of what happens in general. Somewhere in the 
process of solving a first-order partial differential equation, an integration is required to 
remove each partial derivative. At this step an arbitrary function is introduced in the 
solution. This results in an infinite-dimensional solution space. 

In many problems involving partial differential equations it is necessary to select from the 
wealth of solutions a particular solution satisfying one or more auxiliary conditions. As 
might be expected, the nature of these conditions has a profound effect on the existence or 
uniqueness of solutions. A systematic study of such problems will not be attempted in this 
book. Instead, we will treat some special cases to illustrate the ideas introduced in Chapter8 . 


9.2. A first-order partial differential equation with constant coefficients 


Consider the first-order partial differential equation 


(9.4) 3 FY Vy) _ 9, 
Ox Oy 


All the solutions of this equation can be found by geometric considerations. We express the 
left member as a dot product, and write the equation in the form 


(3i + 2j)- V(x, y) = 0. 


This tells us that the gradient vector V/(x, y) is orthogonal to the vector 3 + 2j at each 
point (x, y). But we also know that V(x, y) is orthogonal to the level curves of f. Hence 
these level curves must be straight lines parallel to 3i + 2j. In other words, the level curves 
of f are the lines 

2x — 3y=c. 


Therefore f(x, y) is constant when 2x — 3y is constant. This suggests that 


(9.5) F(x, y) = g(2x — 3y) 


for some function g. 

Now we verify that, for each differentiable function g, the scalar field f defined by (9.5) 
does, indeed, satisfy (9.4). Using the chain rule to compute the partial derivatives of f we 
find 


DOs ay, Ora ag Gu=3y, 
Ox Oy 

3 Fo FL goax — 3y) — 6e'(2x — 3y) =. 
Ox Oy 


Therefore, f satisfies (9.4). 
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Conversely, we can show that every differentiable f which satisfies (9.4) must necessarily 
have the form (9.5) for some g. To do this, we introduce a linear change of variables, 


(9.6) x=Au+ Br, y=Cu+t Dov. 
This transforms f(x, y) into a function of u and v, say 
h(u, v) = f(Au + Br, Cu + De). 
We shall choose the constants A, B, C, D so that / satisfies the simpler equation 


Oh(u,v) 
Ou 


(9.7) 0. 


Then we shall solve this equation and show that f has the required form. 
Using the chain rule we find 


ah _ af ax , Say _ af, af 


= CC. 
Ou Oxdu  Oydu Ox Oy 


Since f satisfies (9.4) we have df/dy = —(3/2)(Af/dx), so the equation for dh/du becomes 


we (43°): 
Ou Ox 2 


Therefore, A will satisfy (9.7) if we choose A = 3C. Taking A = 3 and C = 2 we find 
(9.8) x =3u4+ Br, y=s2ut+ De. 


For this choice of A and C, the function h satisfies (9.7), so A(u, rv) is a function of v alone, 
say 
h(u, v) = g(v) 


for some function g. To express v in terms of x and ¥ we eliminate wu from (9.8) and obtain 
2x — 3y = (2B — 3D)v. Now we choose B and D to make 2B — 3D =1, say B= 2, 
D=1. For this choice the transformation (9.6) is nonsingular; we have r = 2x — 3y, 
and hence 

T(x, ¥) = Atu, cv) = g(t) = g(2x — 3y). 


This shows that every differentiable solution / of (9.4) has the form (9.5). 
Exactly the same type of argument proves the following theorem for first-order equations 
with constant coefficients. 


THEOREM 9.1. Let g be differentiable on R', and let f be the scalar field defined on R® by 
the equation 


(9.9) f(x, ¥) = g(bx — ay), 
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where a and b are constants, not both zero. Then f satisfies the first-order partial differential 
equation 


jg ED. DOS 2% 


9.10 
om” Ox Oy 


everywhere in R?. Conversely, every differentiable solution of (9.10) necessarily has the form 
(9.9) for some g. 


9.3 Exercises 
In this set of exercises you may assume differentiability of all functions under consideration. 


1. Determine that solution of the partial differential equation 


Of (x, Of (x, 
0 oy 
which satisfies the condition f(x, 0) = sin x for all x. 
2. Determine that solution of the partial differential equation 


0 (] 
5 Py) _ 5 ACES Lae 
Ox 0 
which satisfies the conditions f(0, 0) = 0 and D, f(x, 0) = e* for all x. 
3. (a) If u(x, y) = f(xy), prove that u satisfies the partial differential equation 


Ou du 0 
x ax = ay =U. 
Find a solution such that u(x, x) = x*e®’ for all x. 
(b) If v(x, y) = f(x/y) for y ¥ 0, prove that v satisfies the partial-differential equation 


Find a solution such that v(1, 1) = 2 and D,v(x, 1/x) = 1/x for allx 40. 
4. If g(u, v) satisfies the partial differential equation 


d29(u, v) =. 
dudv 


prove that g(u, v) = 9,(u) + %(v), where ¢,(u) is a function of u alone and ¢,(v) is a function 
of v alone. 
5. Assume /‘satisfies the partial differential equation 


“yf, 


ax® “ax ay = ay 


of 


3 — 


0. 


Introduce the linear change of variables, x = Au + Bu, y = Cu + Dv, where A, B, C, D 
are constant, and let g(u, v) = f(Au + Bu, Cu + Dv). Compute nonzero integer values of 


10. 
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A, B, C, D such that g satisfies 02g¢/(0u 0v) = 0. Solve this equation for g and thereby deter- 
mine f- (Assume equality of the mixed partials.) 


. A function u is defined by an equation of the form 


+ 
u(x, y) = wf(* =) . 


Show that uw satisfies a partial differential equation of the form 


> Ou > Ou 
x 2 a 


and find G(x, y). 


. The substitution x = e°, y = e’ converts f(x, y) into g(s, t), where g(s, t) = fle’, e"). If fis 


known to satisfy the partial differential equation 


of anf of of 
2 2 —+yp— = 
i 0x? y oy” *%3 x y oy a 


show that ¢ satisfies the partial-differential equation 


09 Cr 


eae cas 


. Let fbe a scalar field that is differentiable on an open set Sin R”. We say that fis homogeneous 


of degree p over S if 
f(tx) = t?f(x) 


for every t > O and every x in S for which rx € S. For a homogeneous scalar field of degree p 
show that we have 


x: Vf(x) =pf(x)  foreachxinS. 


This is known as Euler’s theorem for homogeneous functions. If x = (x,,...,X,) it can be 
expressed as 
af af 
Xe ae TN ae = p f(X1,.-+5Xn)- 


[Hint: For fixed x, define g(t) = f(rx) and compute 9’(1).] 


. Prove the converse of Euler’s theorem. That is, if f satisfies x - Vf(x) = p f(x) for all x in an 


open set S, then f must be homogeneous of degree p over S. [Hint: For fixed x, define 
&(t) = f(tx) — t”f(x) and compute g’(t).] 

Prove the following extension of Euler’s theorem for homogeneous functions of degree p in 
the 2-dimensional case. (Assume equality of the mixed partials.) 


of anf anf 
re aE ee 
0x2 ao 2xy ox dy + y dy? P(p I) f. 


xX 
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9.4 The one-dimensional wave equation 


Imagine a string of infinite length stretched along the x-axis and allowed to vibrate in the 
xy-plane. We denote by y = f(x, t) the vertical displacement of the string at the point x 
at time ¢. We assume that, at time ¢ = 0, the string is displaced along a prescribed curve, 
y = F(x). An example is shown in Figure 9.1(a). Figures 9.1(b) and (c) show possible 
displacement curves for later values of t. We regard the displacement f(x, t) as an unknown 
function of x and ¢ to be determined. A mathematical model for this problem (suggested 
by physical considerations which we shall not discuss here) is the partial differential equation 


Of _ 29f 
or” ax?’ 


where c is a positive constant depending on the physical characteristics of the string. This 
equation is called the one-dimensional wave equation. We will solve this equation subject to 
certain auxiliary conditions. 


y 


y = f(x, 0) y = fix, }) y =flx, 1) 


x x 
a 6 -f =f 0° 1 Sa 0. 4 


(a)t =0 (b)t = 3 (c)t=1 
FIGuRE 9.1 The displacement curve y = f(x, t) shown for various values of ¢. 


Since the initial displacement is the prescribed curve y = F(x), we seek a solution 
satisfying the condition 


I(x, 0) = F(x). 


We also assume that dy/dr, the velocity of the vertical displacement, is prescribed at time 
t= 0, say 


Dof (x, 0) = G(x), 


where G Is a given function. It seems reasonable to expect that this information should 
suffice to determine the subsequent motion of the string. We will show that, indeed, this is 
true by determining the function fin terms of Fand G. The solution is expressed in a form 
given by Jean d’Alembert (1717-1783), a French mathematician and philosopher. 


THEOREM 9.2. D’ALEMBERT’S SOLUTION OF THE WAVE EQUATION. Let F and G be given 
functions such that G is differentiable and F is twice differentiable on R'. Then the function f 
given by the formula 


(9.11) f(x, )= F(x + ct) + F(x — et) 4 i [6 ds 


2 2c 


xr—C 
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satisfies the one-dimensional wave equation 


of of 
9.12 — = ¢ — 
om) ort’ : ex? 
and the initial conditions 
(9.13) f(x, 0) = F(x), = Dy f(x, 0) = G(x). 


Conversely, any function f with equal mixed partials which satisfies (9.12) and (9.13) neces- 
sarily has the form (9.11). 


Proof. It is a straightforward exercise to verify that the function f given by (9.11) 
satisfies the wave equation and the given initial conditions. This verification is left to the 
reader. We shall prove the converse. 

One way to proceed is to assume that fis a solution of the wave equation, introduce a 
linear change of variables, 


x = Aust Bo, t= Cu+t+ Dov, 
which transforms f(x, ¢) into a function of u and v, say 
g(u, v) = f(Au + Bv, Cu + Dov), 
and choose the constants A, B, C, D so that g satisfies the simpler equation 


2 
TE a. 
Ou Ov 


Solving this equation for g we find that g(u, v) = ,(u) + ¢,(v), where ¢,(u) is a function 
of u alone and ¢,(v) is a function of v alone. The constants A, B, C, D can be chosen so 
thatu = x+ct,v = x — ct, from which we obtain 


(9.14) f(x, t) = G(x + ct) + p(x — ct). 


Then we use the initial conditions (9.13) to determine the functions g, and qg, in terms of 


the given functions F and G. 
We will obtain (9.14) by another method which makes use of Theorem 9.1 and avoids the 


change of variables. First we rewrite the wave equation in the form 
(9.15) Ly(Lef ) = 0, 
where L, and L, are the first-order linear differential operators given by 


0 0 


= — — ¢ — 


‘Ot Ox” 


Let f be a solution of (9.15) and let 


u(x, t) = Lef(x, 1). 


290 Applications of differential calculus 


Equation (9.15) states that wu satisfies the first-order equation L,(u) =0. Hence, by 
Theorem 9.1 we have 


u(x,t) = o(x + cr) 


for some function g. Let ® be any primitive of g, say B(y) = ff ¢(s) ds, and let 
1 
v(x, t) = — D(x + ct). 
2c 


We will show that L,(v) = L,(f). We have 


— sO + ct) and ~ = =O + ct), 
SO 
Ov Ov , 
lea oe + ct) = g(x + ct) = u(x, t) = Lf. 
x 


In other words, the difference f — v satisfies the first-order equation 
L,(f— v) =0. 


By Theorem 9.1 we must have f(x, t) — v(x, t) = p(x — ct) for some function y. There- 
fore 


OR OO Cae ee ee 5 (x a eae eee 


l 
This proves (9.14) with g, = e ® and 9, = y. 


Now we use the initial conditions (9.13) to determine the functions g, and q, in terms of 
the given functions F and G. The relation f(x, 0) = F(x) implies 


(9.16) P(x) + 9(x) = F(x). 
The other initial condition, Daf(x, 0) = G(x), implies 
(9.17) eyi(x) — epi(x) = G(x). 
Differentiating (9.16) we obtain 

(9.18) P(x) + p(x) = F(x). 


Solving (9.17) and (9.18) for g{(x) and ,(x) we find 


gi) = Fs) + 5 G(s), p(x) = F(x) = z G(x). 
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Integrating these relations we get 


p(x) — 9,(0) = foe) + ES [60 ds, 


2 2c Jo 
Pax) — (0) = wr — = | 6) ds. 


In the first equation we replace x by x + cf; in the second equation we replace x by 
x — ct. Then we add the two resulting equations and use the fact that ¢,(0) + (0) = 
F(0) to obtain 


zrt+et 


— Fete + Feet) 1 G(s) ds. 


f(x, 1) = pix + ct) + pax — ct) = : 
C Jax-ct 


This completes the proof. 
EXAMPLE. Assume the initial displacement is given by the formula 
, for —-l<x<l, 


F(x) = 
©) 0 for |x| >1. 


y = f(x, 9) = F(x) 


Sh By ed 0 l 2 3 4 
(a)t = 0 (b)t = 2 


FiGuRE 9.2 A solution of the wave equation shown for t = 0 and ¢ = 2. 


The graph of F is shown in Figures 9.1(a) and 9.2(a). Suppose that the initial velocity 
G(x) = Ofor all x. Then the resulting solution of the wave equation Is given by the formula 


F(x + ct) + F(x — ct 
f(x, 1) = SEDATE 
2 
Figures 9.1 and 9.2 show the curve y = f(x, ¢) for various values of t. The figures illustrate 


that the solution of the wave equation is a combination of two standing waves, one traveling 
to the right, the other to the left, each with speed c. 


Further examples illustrating the use of the chain rule in the study of partial differential 
equations are given in the next set of exercises. 
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9.5 Exercises 


— 


wo 


In this set of exercises you may assume differentiability of all functions under consideration. 


. If k is a positive constant and g(x, t) = 3x//kr, let 


f(x, t) = (a en” du. 


0 


fe B® any Fo B 


(a) Show that 


() 
ax Ox = Or or 
(b) Show that f satisfies the partial differential equation 


2 0 
k i ih 
Or 


7a (the heat equation). 
x 


. Consider a scalar field f defined in R? such that f(x, y) depends only on the distance r of (x, y) 


from the origin, say f(x, y) =g(r), where r = (x? + y®)%. 
(a) Prove that for (x, y) ¥ (0, 0) we have 


af Ct I. , 
re T 3 ="s + ¢ (r). 


(b) Now assume further that fsatisfies Laplace’s equation, 


32 92 
% fy, 
Ox oy” 


for all (x, y) # (0,0). Use part (a) to prove that f(x, y) = alog (x? + y?) + b for (x, y) # 
(0, 0), where a and 6b are constants. 


. Repeat Exercise 2 for the n-dimensional case, where n > 3. That is, assume that f(x) = 


f(%15-++5%n) =r), where r = ||x||. Show that 
o"f ef in-l, ; 
oe me ETE 


forx # O. If fsatisfies the n-dimensional Laplace equation, 


he Qe? 


ax? Ox 


for all x # O, deduce that f(x) = a ||x||?-" + 5 for x # O, where a, b are constants. 


Note: The linear operator V? defined by the equation 


al ay 


n 


is called the n-dimensional Laplacian. 
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4. Two-dimensional Laplacian in polar coordinates. The introduction of polar coordinates x = 
rcos 0, y = rsin @, converts f(x, y) into g(r, 6). Verify the following formulas: 


dg\? —s 1 / ag\? 
(a) || V(r cos 0, rsin 6)||? = (2) + =(3) : 


wh ef ag 1 ag 1 a 
Gat aye oe OOF Par 


5. Three-dimensional Laplacian in spherical coordinates. The introduction of spherical coordinates 
x = pcos Osin g, y = psin Osing, Z = pcos 9, 


transforms f(x, y, z) to F(p, 9, y). This exercise shows how to express the Laplacian V?f in 
terms of partial derivatives of F. 

(a) First introduce polar coordinates x = rcos 8, y = rsin 0 to transform f(x, y, z)tog(r, 9, z). 
Use Exercise 4 to show that 


cepa 4 1s ae, He 
ar2 | r2 062 ' yp ar | az?" 


(b) Now transform g(r, 9, z) to F(p, 9, y) by taking z = pcos y, r = psin gy. Note that, 
except for a change in notation, this transformation is the same as that used in part (a). Deduce 
that 
- a 2 aF 1 OF cos OF 1 oF 
f= 5 pop pag? p*sing ém  p*sing 002— 


6. This exercise shows how Legendre’s differential equation arises when we seek solutions of 
Laplace’s equation having a special form. Let fbe a scalar field satisfying the three-dimensional 
Laplace equation, V2f = 0. Introduce spherical coordinates as in Exercise 5 and let F(p, 0, y) = 
f(x, y, 2). 

(a) Suppose we seek solutions f of Laplace’s equation such that F(p, 8, g) is independent of 0 
and has the special form F(p, 9, y) = p"G(g). Show that f satisfies Laplace’s equation if G 
satisfies the second-order equation 


2 


de +n(n+1)G =0. 


dG 
+ cot y— We 


(b) The change of variable x = cos » (y = arccosx, —1 < x < 1) transforms G(¢) to g(x). 
Show that g satisfies the Legendre equation 


d"g dg 

— x7) —— —2x— = (0. 
(1 x) Te xa tata + Dg 0 
7. Two-dimensional wave equation. A thin flexible membrane is stretched over the xy-plane and 
allowed to vibrate. Let z = f(x, y, t) denote the vertical displacement of the membrane at the 


point (x, y) at time ¢. Physical considerations suggest that fsatisfies the two-dimensional wave 


equation, 
anf (33 anf =) 
—==Cc + 


or? - ox? dy” 
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where c is a positive constant depending on the physical characteristics of the membrane. This 
exercise reveals a connection between this equation and Bessel’s differential equation. 

(a) Introduce polar coordinates x = rcos 6, y = rsin 0, and let F(r, 6, t) = f(rcos 6, rsin 6,1). 
If f satisfies the wave equation show that F satisfies the equation 


Q°F ‘ o*F 1 oF 1 OF 
Be” Noe peabes © 4 Bek 


(b) If F(r, 6, t) is independent of 6, say F(r, 9, t) = g(r, t) the equation in (a) simplifies to 


ep pe 
or? r Or 


Now let ¢ be a solution such that g(r, t) factors into a function of r times a function of ¢, say 
p(r,t) = R(r)T(t). Show that each of the functions R and T satisfies an ordinary linear differ- 
ential equation of second order. 


(c) If the function 7 in part (b) is periodic with period 27/c, show that R satisfies the Bessel 
equation ?R” + rR’ +r?R=0. 


9.6 Derivatives of functions defined implicitly 


Some surfaces in 3-space are described by Cartesian equations of the form 
F(x, y,z)=0. 


An equation like this is said to provide an implicit representation of the surface. For 
example, the equation x? + y? + z? — 1 = 0 represents the surface of a unit sphere with 
center at the origin. Sometimes it is possible to solve the equation F(x, y, z) = 0 for one 
of the variables in terms of the other two, say for zin terms of x and y. This leads to one or 
more equations of the form 


z = f(x, y). 


For the sphere we have two solutions, 
z=V1l—x—y and z= —V1— x? — y?, 


one representing the upper hemisphere, the other the lower hemisphere. 

In the general case it may not be an easy matter to obtain an explicit formula for z in 
terms of x and y. For example, there is no easy method for solving for z in the equation 
ytxz+22—e7—4=0. Nevertheless, a judicious use of the chain rule makes it 
possible to deduce various properties of the partial derivatives of/0x and @f/dy without an 
explicit knowledge of f(x, y). The procedure is described in this section. 

We assume that there is a function f(x, y) such that 


(9.19) F[x, y, f(x, y)] = 0 


for all (x, y) in some open set S, although we may not have explicit formulas for calculating 
I(x, y). We describe this by saying that the equation F(x, y, z) = 0 defines z implicitly as a 
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function of x and y, and we write 
z= f(x; y). 


Now we introduce an auxiliary function g defined on S as follows: 


2(x, y) = F[x, y, f(x, y)]. 


Equation (9.19) states that g(x, y) = 0 on S; hence the partial derivatives dg/dx and 
dg/dy are also 0 on S. But we can also compute these partial derivatives by the chain rule. 
To do this we write 


g(x, y) = F[u,(x, y), U(x, y), Us(X, y)I, 


where u(x, y) = x, u(x, y) = y, and u,(x, y) = f(x, y). The chain rule gives us the 
formulas 


Og =<: ps 225 yp mn DF 22 — Og Ou, Ou, pes 


Ox Ox Ox Ox oy Oy Oy Oy” 


where each partial derivative D,F is to be evaluated at (x, y, f(x, y)). Since we have 


II 
aX 
2 |= 
oO 
| 
a 
IS 
= 
Q. 
a 
oQ 
| 
oO 


the first of the foregoing equations becomes 
D,F + DF of = 
Ox 
Solving this for df/ex we obtain 
(9.20) of eo D,F{x, yr f(x, y)] 
Ox D3F[x, y, f(x, y)] 


at those points at which D3,F'[x, y, f(x, y)] A 0. By a similar argument we obtain a corre- 
sponding formula for df/dy: 


(9.21) of _ _ DF Ix, y, f(x WI 


Oy D3F[x, y, f(x, y)] 


at those points at which D3F[x, y, f(x, y)] # 0. These formulas are usually written more 
briefly as follows: 


of OF /dx of _ ‘OF jy 
Ox OF/dz° ay OF /dz 


EXAMPLE. Assume that the equation y? + xz + z? — e® — c = O defines z as a function 
of x and y, say z = f(x, y). Find a value of the constant c such that /(0, e) = 2, and 
compute the partial derivatives 0f/0x and @f/dy at the point (x, y) = (0, e). 
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Solution. Whenx =0,y = e,andz = 2, the equation becomes e? + 4 — e? —c = 0, 
and this is satisfied by c=4. Let F(x, y,z) = y? + xz + 22? —e*—4. From (9.20) 
and (9.21) we have 


When x = 0, y =e, and z=2 we find df/dx = 2/(e? — 4) and df/dy = 2e/(e® — 4). 
Note that we were able to compute the partial derivatives Of/dx and Of/dy using only the 
value of f(x, y) at the single point (0, e). 


The foregoing discussion can be extended to functions of more than two variables. 


THEOREM 9.3. Let F be a scalar field differentiable on an open set T in R". Assume that 
the equation 
F(5834%;) = 9 


defines x, implicitly as a differentiable function of x,,..., Xn—15 SAY 
Xn = f(x, ae) Xia) 
for all points (X,,... , X,_,) in some open set SinR". Then for eachk = 1,2,...,n—1, 


the partial derivative D,f is given by the formula 


D,F 
9.22 D,f=- —- 
(9.22) ut DF 
at those points at which D,F #0. The partial derivatives D,F and D,F which appear in 
(9.22) are to be evaluated at the point (X,, X2,.--5Xn-1sf (X15 +++ 5 Xn-1))- 


The proof is a direct extension of the argument used to derive Equations (9.20) and 
(9.21) and is left to the reader. 

The discussion can be generalized in another way. Suppose we have two surfaces with 
the following implicit representations: 


(9.23) F(x, y,z) = 0, G(x, y,z) =0. 


If these surfaces intersect along a curve C, it may be possible to obtain a parametric 
representation of C by solving the two equations in (9.23) simultaneously for two of the 
variables in terms of the third, say for x and y in terms of z. Let us suppose that it is 
possible to solve for x and y and that solutions are given by the equations 


x=X(z), y= YV(2) 


for all z in some open interval (a, 6). Then when x and y are replaced by X(z) and Y(z), 
respectively, the two equations in (9.23) are identically satisfied. That is, we can write 
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F[X(z), Y(z), z] =0 and G[X(z), Y(z), z] = 0 for all z in (a,b). Again, by using the 


chain rule, we can compute the derivatives X ‘(z) and Y ‘(z) without an explicit knowledge of 
X(z) and Y(z). To do this we introduce new functions fand g by means of the eyuations 


f(z) = F[X(2), Y(2), Z] and 2(z) = G[X(z), Y(2), z]. 


Then f(z) = g(z) = 0 for every z in (a,b) and hence the derivatives f’(z) and g’(z) are 
also zero on (a, b). By the chain rule these derivatives are given by the formula 


fQa=Fx@+Fvea+s. g'(2) = 2 2 x'@) +S oye + S 


Since f(z) and g’(z) are both zero we can determine X '(z) and Y'(z) by solving the follow- 
ing pair of simultaneous /inear equations: 


OF OF 


a NO* GO ec 
0G 0G 
Fy ex@+h ~V@=—s. 


At those points at which the determinant of the system is not zero, these equations have a 
unique solution which can be expressed as follows, using Cramer’s rule: 


OF OF OF OF 

Oz Oy Ox az 

0G aG 0G 9G 

; Oz Oy Ox dz 
(9.24) X (z) = — ———— Y(z) = - —————_.. 
OF OF OF OF 

Ox Oy Ox dy 

0G 0G 0G 0G 

Ox Oy Ox dy 


The determinants which appear in (9.24) are determinants of Jacobian matrices and are 
called Jacobian determinants. A special notation is often used to denote Jacobian deter- 


minants. We write 


Di i neg 

OX, OX, OX, 
ee A oe ae | 
O( Gis enn ks) 


Ox, OX, Ox, 


298 Applications of differential calculus 


In this notation, the formulas in (9.24) can be expressed more briefly in the form 


_ HF, G)/ay, z) 


_ _ O(F, G)/o(z, x) 
AF, G)/A(x, y) ” 


O) = aR, Gya(x, ») 


(9.25) X'(z) 
(The minus sign has been incorporated into the numerators by interchanging the columns.) 

The method can be extended to treat more general situations in which m equations in n 
variables are given, where n > m and we solve for m of the variables in terms of the 
remaining n — m variables. The partial derivatives of the new functions so defined can be 
expressed as quotients of Jacobian determinants, generalizing (9.25). An example with 
m = 2 and n = 4 is described in Exercise 3 of Section 9.8. 


9.7 Worked examples 


In this section we illustrate some of the concepts of the foregoing section by solving 
various types of problems dealing with functions defined implicitly. 


EXAMPLE |. Assume that the equation g(x, y) =0 determines y as a differentiable 
function of x, say y = Y(x) for all x in some open interval (a, b). Express the derivative 
Y’(x) in terms of the partial derivatives of g. 


Solution. Let G(x) = g[x, Y(x)] for x in (a, b). Then the equation g(x, y) = 0 implies 
G(x) = 0in (a, 5). By the chain rule we have 


G'(x) = a 1 + Og Y’(x), 
Ox Oy 
from which we obtain 
_ dglox 


(9.26) Y(x) = Deldy 


at those points x in (a, b) at which dg/dy # 0. The partial derivatives dg/0x and dg/dy 
are given by the formulas dg/dx = D,g[x, Y(x)] and dg/dy = Dog|x, Y(x)]. 


EXAMPLE 2. When y is eliminated from the two equations z = f(x, y) and g(x, y) = 0, 
the result can be expressed in the form z = h(x). Express the derivative h’(x) in terms of 
the partial derivatives of fand g. 


Solution. Let us assume that the equation g(x, y) = 0 may be solved for y in terms of 
x and that a solution is given by y = Y(x) for all x in some open interval (a, b). Then 
the function h is given by the formula 


h(x) = f[x, Y(x)] if xe€(a,b). 


Applying the chain rule we have 


af. af 


pa ga 
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Using Equation (9.26) of Example 1 we obtain the formula 


ogof of dg 


The partial derivatives on the right are to be evaluated at the point (x, Y(x)). Note that 
the numerator can also be expressed as a Jacobian determinant, giving us 


a(f, g)/0(x, y) | 
Og/oy 


EXAMPLE 3. The two equations 2x = v? — uw? and y = uv define u and v as functions 
of x and y. Find formulas for du/0x, du/dy, dv/dx, dv/dy. 


h'(x) = 


Solution. If we hold y fixed and differentiate the two equations in question with respect 
to x, remembering that u and v are functions of x and y, we obtain 


2 = 2p 28 _ ay, OU and ea ae. 
Ox Ox Ox Ox 


Solving these simultaneously for du/dx and dv/dx we find 


On the other hand, if we hold x fixed and differentiate the two given equations with respect 
to y we obtain the equations 
Ov Ou Ov Ou 


0 = 20 — — 2u — and l=u—-+v0-—. 
Oy Oy Oy dy 


Solving these simultaneously we find 


Ou _ v d Ov _ u 
a 2 2 a 7 8 2° 
Oy uét+ou Oy ut +o 


EXAMPLE 4. Let u be defined as a function of x and y by means of the equation 
u= F(x + u, yu). 
Find du/dx and du/dy in terms of the partial derivatives of F. 


Solution. Suppose that u = g(x, y) for all (x, y) in some open set S. Substituting 
g(x, y) for u in the original equation we must have 


(9.27) g(x, y) = Fluy(x, y), u(x, y)], 
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where u,(x, y) = x + g(x, y) and u,(x, y) = y g(x, y). Now we hold y fixed and differ- 
entiate both sides of (9.27) with respect to x, using the chain rule on the right, to obtain 


(9.28) a. — if — + D,F — ° 
x 


But du,/0x = 1 + dg/ox, and du,/dx = y dg/dx. Hence (9.28) becomes 
Og _ 0g Og 
= D,F- {1 DF : 
Ox ( Bs Ox ed - (» 5.) 
Solving this equation for dg/0x (and writing du/dx for dg/dx) we obtain 


Ou —D,F 


dx DF +yD.F—1- 
In a similar way we find 


Og Ou, Ou, dg (v2 Og ). 
— = D,F — + DF — = D\F > + DAF ly = + a(x, 
ay ay ay y y ay 9(x, y) 
This leads to the equation 
Ou = — g(x, y) D,F 


dy D,F+yD.F-1_— 
The partial derivatives D,F and D,F are to be evaluated at the point (x + g(x, y), y g(x, y)). 
EXAMPLE 5. When u is eliminated from the two equations x = u + v and y = uv*, we 
get an equation of the form F(x, y, v) = 0 which defines v implicitly as a function of x 


and y, say v = h(x, y). Prove that 


oh _ __—iA ) 
Ox 3h(x, y) — 2x 


and find a similar formula for dh/dy. 
Solution. Eliminating u from the two given equations, we obtain the relation 
x? —v—y=0. 
Let F be the function defined by the equation 
F(x, y, v) = xv? — v — y. 


The discussion in Section 9.6 is now applicable and we can write 


(9.29) dh —— OF /dx and oh _—oOF/0y | 
Ox OF /dv Oy OF [ov 
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But 0F/0x = v*, OF/dv = 2xv — 3v’, and OF/dy = —1. Hence the equations in (9.29) 
become 

Oh v _ v h(x, y) 

ax 2xv — 3v? 2x —3v  3h(x, y) — 2x 
and 

Oh _ ~ 1 


ay 2xv — 3v2 —-2xh(x, y) — 3h7(x, y) 


EXAMPLE 6. The equation F(x, y, z) = 0 defines z implicitly as a function of x and y, 
say z = f(x, y). Assuming that 0?F/(0x dz) = 0?F/(0z 0x), show that 


ww 2, ee) ox) ~ "oa ee) (6d) * Ce) (oe) 


Ox? fe 3 
) 


where the partial derivatives on the right are to be evaluated at (x, y, f(x, y)). 


Solution. By Equation (9.20) of Section 9.6 we have 


of _ oF [ox 
Ox OF/dz 


(9.31) 


We must remember that this quotient really means 


= D,FIx, y, F(x, y)} 
D3F[x, WS (x, y)] 


Let us introduce G(x, y) = D,F[x, y, f(x, y)] and A(x, y) = D3F[x, y, f(x, y)]. Our 
object is to evaluate the partial derivative with respect to x of the quotient 


of _ _— Gx, y) 


Ox H(x, y)” 
holding y fixed. The rule for differentiating quotients gives us 


0G oH 
at ae Be 
9.32 —_—S = ES, 
( ) Ox? H? 


Since G and H are composite functions, we use the chain rule to compute the partial 
derivatives 0G/Ox and 0H/dx. For 0G/dx we have 


OG ra] 
2G _ D(D,F)-1 + DAD,F) +0 + D,(D,F) = 
Ox Ox 


OF, OF a 
Ox? @z0xdx 
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Similarly, we find 


oH rs] 
Mf 2D (pei Dey 0 DDFS 
Ox Ox 
_@F | er af 
Oxdz Oz dx 


Substituting these in (9.32) and replacing Of/dx by the quotient in (9.31) we obtain the 
formula in (9.30). 


9.8 


Exercises 


In the exercises in this section you may assume the existence and continuity of all derivatives 
under consideration. 


1. 


The two equations x + y = uv and xy = u — v determine x and y implicitly as functions of 
uand v, say x = X(u,v) and y = Y(u,v). Show that 0X/ou = (xv — 1)/(x — y) if x Ay, 
and find similar formulas for 0X/0v, 0Y/0u, 0Y/ dv. 


. The two equations x + y = uv and xy = u — v determine x and v as functions of u and y, 


say x = X(u, y) and v = V(u, y). Show that 0X/eu = (u + v)/(1 + yu) if 1 + yu #0, and 
find similar formulas for 0X/éy, @V/eu, OV] dy. 


. The two equations F(x, y, u,v) =0 and G(x, y, u,v) = 0 determine x and y implicitly as 


functions of u and v, say x = X(u,v) and y = Y(u,v). Show that 


aX  a(F, G)/a(y, wu) 


du a(F, G)/ (x, y) 


at points at which the Jacobian 0(F, G)/0(x, y) ¥ 0, and find similar formulas for the partial 
derivatives 0X/0v, 0Y/0u, and @Y/ dv. 


. The intersection of the two surfaces given by the Cartesian equations 2x? + 3y? — z* = 25 


and x* + y” = z® contains a curve C passing through the point P = (iT, 3,4). These 
equations may be solved for x and y in terms of z to give a parametric representation of C 
with z as parameter. 

(a) Find a unit tangent vector T to C at the point P without using an explicit knowledge of 
the parametric representation. 

(b) Check the result in part (a) by determining a parametric representation of C with z as 
parameter. 


. The three equations F(u,v) =0, u = xy, and v = ./x* + z* define a surface in xyz-space. 


Find a normal vector to this surface at the pointx =1, y=1,z= J3 if it is known that 
D,F(U, 2) = 1 and D,F(1, 2) = 2. 


. The three equations 


x® — ycos (wv) +z? =0, 
x? + y® — sin (uv) + 22? = 2, 
xy —sinucosv +z =0, 


define x, y, and z as functions of u and v. Compute the partial derivatives 0x/0u and 0x/9dv at 
the pointx =y=l1,u=7/2,v =0,z=0. 
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7. The equation f(y/x, z/x) = 0 defines z implicitly as a function of x and y, say z = g(x, y). 
Show that 
og og 
xe Ty ay = g(x, y) 
at those points at which D,f[y/x, g(x, y)/x] is not zero. 

8. Let F be a real-valued function of two real variables and assume that the partial derivatives 
D,F and D.F are never zero. Let u be another real-valued function of two real variables such 
that the partial derivatives ou/@x and du/ dy are related by the equation F(du/ 0x, Ou/ oy) = 0. 
Prove that a constant n exists such that 


e2u 0*y a2y \" 
ax? ay? ~ \ ax dy : 


and find n. Assume that 02u/(@x dy) = 0u/(dy ax). 

9. The equation x +z +(y +z)? =6 defines z implicitly as a function of x and y, say 
z = f(x, y). Compute the partial derivatives of/ax, af/dy, and 0%f/(ex dy) in terms of x, 
y, and z. 

10. The equation sin (x + y) + sin (y + z) = 1 defines z implicitly as a function of x and y, say 
z = f(x, y). Compute the second derivative D, .f in terms of x, y, and z. 

11. The equation F(x + y + z, x? + y? + z*) =0 defines z implicitly as a function of x and y, 
say z = f(x, y). Determine the partial derivatives of/0x and af/0y in terms of x, y, z and the 
partial derivatives D,F and D,F. 

12. Let fand g be functions of one real variable and define F(x, y) = f[x + g(y)]. Find formulas 
for all the partial derivatives of F of first and second order, expressed in terms of the derivatives 
of fand g. Verify the relation 


——— memes SE ges ee 


9.9 Maxima, minima, and saddle points 


A surface that is described explicitly by an equation of the form z = f(x, y) can be 
thought of as a level surface of the scalar field F defined by the equation 


F(x, y, z) = f(x, y) — Zz. 


If fis differentiable, the gradient of this field is given by the vector 


A linear equation for the tangent plane at a point P; = (x,, yi, 21) can be written in the form 


Z— Zz, = A(x — x) + Bly — yi); 
where 
A= D, f (X15 1) and B= Dz f (X15 y1)- 


When both coefficients A and B are zero, the point P, is called a stationary point of the 
surface and the point (x,, y) is called a stationary point or a critical point of the function /- 
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The tangent plane is horizontal at a stationary point. The stationary points of a surface are 
usually classified into three categories: maxima, minima, and saddle points. If the surface 
is thought of as a mountain landscape, these categories correspond, respectively, to 
mountain tops, bottoms of valleys, and mountain passes. 

The concepts of maxima, minima, and saddle points can be introduced for arbitrary scalar 
fields defined on subsets of R”. 


DEFINITION. A scalar field f is said to have an absolute maximum at a point a of a set S 
in R” if 


(9.33) f(x) <f@ 


for all x in S. The number f(a) is called the absolute maximum value of fon S. The function f 
is said to have a relative maximum at a if the inequality in (9.33) is satisfied for every x in 
some n-ball B(a) lying in S. 


In other words, a relative maximum at ais the absolute maximum in some neighborhood 
of a. The terms absolute minimum and relative minimum are defined in an analogous 
fashion, using the inequality opposite to that in (9.33). The adjectives global and local are 
sometimes used in place of absolute and relative, respectively. 


DEFINITION. A number which is either a relative maximum or a relative minimum of f is 
called an extremum of f. 


If fhas an extremum at an interior point a and is differentiable there, then all first-order 
partial derivatives D,f(a),..., D,f(a) must be zero. In other words, Vf(a) = O. (This 
is easily proved by holding each component fixed and reducing the problem to the one- 
dimensional case.) In the case n = 2, this means that there is a horizontal tangent plane 
to the surface z = f(x, y) at the point (a, f(a)). On the other hand, it is easy to find examples 
in which the vanishing of all partial derivatives at a does not necessarily imply an extremum 
at a. This occurs at the so-called saddle points which are defined as follows. 


DEFINITION. Assume f is differentiable ata. If Vf(a) = O the point ais called a stationary 
point of f. A stationary point is called a saddle point if every n-ball B(a) contains points x 
such that f(x) < f(a) and other points such that f(x) > f(a). 


The situation is somewhat analogous to the one-dimensional case in which stationary 
points of a function are classified as maxima, minima, and points of inflection. The 
following examples illustrate several types of stationary points. In each case the stationary 
point in question is at the origin. 


EXAMPLE 1. Relative maximum. z = f(x, y) = 2 — x? — y®. This surface is a parabo- 
loid of revolution. In the vicinity of the origin it has the shape shown in Figure 9.3(a). Its 
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(ajz=2-— x — y? (b) Level curves: x? + y?>=c 


Example 1. Relative maximum at the origin. 


Z 


. Y 
| 


()z=xr+y 


Example 2. Relative minimum at the origin. 


FIGURE 9.3. Examples | and 2. 


level curves are circles, some of which are shown in Figure 9.3(b). Since f(x, y) = 
2 — (x27 + y*?) < 2 = f(0, 0) for all (x, y), it follows that fnot only has a relative maximum 


at (0,0) but also an absolute maximum there. Both partial derivatives 0f/0x and df/dy 
vanish at the origin. 


EXAMPLE 2. Relative minimum. z = f(x, y) = x? + y®. This example, another parabo- 
loid of revolution, is essentially the same as Example 1, except that there is a minimum at 
the origin rather than a maximum. The appearance of the surface near the origin is 
illustrated in Figure 9.3(c) and some of the level curves are shown in Figure 9.3(b). 
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EXAMPLE 3. Saddle point. z = f(x, y) = xy. This surface is a hyperbolic paraboloid. 
Near the origin the surface is saddle shaped, as shown in Figure 9.4(a). Both partial 
derivatives 0f/0x and Of/Oy are zero at the origin but there is neither a relative maximum 
nor a relative minimum there. In fact, for points (x, y) in the first or third quadrants, 
x and y have the same sign, giving us f(x, y) > 0 = f(0, 0), whereas for points in the 
second and fourth quadrants x and y have opposite signs, giving us f(x, y) < 0 = f(0, 0). 
Therefore, in every neighborhood of the origin there are points at which the function is 
less than f(0, 0) and points at which the function exceeds /(0, 0), so the origin is a saddle 


Zz y 


JAN. 
‘VW 


(a) Z = xy (b) Level curves: xy = ¢ 


FiGurE 9.4 Example 3. Saddle point at the origin. 


point. The presence of the saddle point is also revealed by Figure 9.4(b), which shows 
some of the level curves near (0,0). These are hyperbolas having the x- and y-axes as 
asymptotes. 


EXAMPLE 4. Saddle point. z = f(x, y) = x* — 3xy®. Near the origin, this surface has 
the appearance of a mountain pass in the vicinity of three peaks. This surface, sometimes 
referred to as a “monkey saddle,”’ is shown in Figure 9.5(a). Some of the level curves 
are illustrated in Figure 9.5(b). It is clear that there is a saddle point at the origin. 


EXAMPLE 5. Relative minimum. z = f(x, y) = x*y®. This surface has the appearance of 
a valley surrounded by four mountains, as suggested by Figure 9.6(a). There is an absolute 
minimum at the origin, since f(x, y) > f(0, 0) for all (x, y). The level curves [shown in 
Figure 9.6(b)] are hyperbolas having the x- and y-axes as asymptotes. Note that these level 
curves are similar to those in Example 3. In this case, however, the function assumes only 
nonnegative values on all its level curves. 


EXAMPLE 6. Relative maximum. z = f(x, y) = 1— x®. In this case the surface is a 
cylinder with generators parallel to the y-axis, as shown in Figure 9.7(a). Cross sections 
cut by planes parallel to the x-axis are parabolas. There is obviously an absolute maximum 
at the origin because f(x, y) = 1 — x? < 1 = f(0, 0) for all (x, y). The level curves form a 
family of parallel straight lines as shown in Figure 9.7(b). 


(a)z =x? — 3xy’. (b) Level curves: x? — 3xy? =c. 


FIGURE 9.5 Example 4. Saddle point at the origin. 
y 


x 
x 
(a) z= x’y? (b) Level curves: x’?y? = c 
FiGuRE 9.6 Example 5. Relative minimum at the origin. 
Zz y 
Tangent plane at (0,0,1) 
x 


(a)z=1—~ (b) Level curves: 1 — x? =c 
FIGURE 9.7 Example 6. Relative maximum at the origin. 
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9.10 Second-order Taylor formula for scalar fields 


If a differentiable scalar field f has a stationary point at a, the nature of the stationary 
point is determined by the algebraic sign of the difference f(x) — f(a) for x near a. If 
x =a+ty, we have the first-order Taylor formula 


fiat y)—f(@ = Vf(@):y + lyll E(@y), where E(a,y)>0 asyO. 
At a stationary point, V/(a) = O and the Taylor formula becomes 
f(a + y) —f(@ = lly|| E@ y). 


To determine the algebraic sign of f(a + y) — f(a) we need more information about the 
error term ||y|| E(a, y). The next theorem shows that if f has continuous second-order 
partial derivatives at a, the error term is equal to a quadratic form, 


1» S Dufl@yy, 


a a 


plus a term of smaller order than ||y||?. The coefficients of the quadratic form are the 
second-order partial derivatives D,,;f = D,;(D,f), evaluated at a. The n x n matrix of 
second-order derivatives D,,; f(x) is called the Hessian matrix} and is denoted by H(x). 
Thus, we have 


H(x) = [Dif (x) i jan 


whenever the derivatives exist. The quadratic form can be written more simply in matrix 
notation as follows: 


Yd Piuf@v, = vHoy, 


iM: 


where y = (y1,---+, Yn) 18S considered as a 1 X n row matrix, and y’ is its transpose, an 
n X 1 column matrix. When the partial derivatives D,;f are continuous we have D,,;f = 
D,f and the matrix H(a) is symmetric. 

Taylor’s formula, giving a quadratic approximation to f(a + y) — f(a), now takes the 
following form. 


THEOREM 9.4, SECOND-ORDER TAYLOR FORMULA FOR SCALAR FIELDS. Let f be a scalar 
field with continuous second-order partial derivatives D;,f in an n-ball B(a). Then for all 
y in R” such that a + y € B(a) we have 


(9.34) f(a+y) —f(@ =Vf(a):-yt+ | vH(a + cy)y’, where 0<c<l. 


+t Named for Ludwig Otto Hesse (1811-1874), a German mathematician who made many contributions to 
the theory of surfaces. 
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This can also be written in the form 
(9.35) fla +») — f@ =Vf@)-y + = yHlay' + ly!" Ela, y), 
where E,(a, y) >O0asy—>O. 
Proof. Keep y fixed and define g(u) for real u by the equation 
g(u) = f(a + uy) for -—-l<u<l. 


Then f(a + y) — f(a) = g(1) — g(0). We will prove the theorem by applying the second- 
order Taylor formula to g on the interval [0, 1]. We obtain 


(9.36) g(1) — g(0) = 2'(0) Hy 58"); where 0<c <1. 


Here we have used Lagrange’s form of the remainder (see Section 7.7 of Volume I). 
Since g is a composite function given by g(u) = f[r(u)], where r(u) = a + uy, we can 
compute its derivative by the chain rule. We have r’(u) = y so the chain rule gives us 


g'(u) = Vf[r(u)] ru) = Vflru)) -y = =D, Fry]; 


provided r(u) € B(a). In particular, g’(0) = Vf(a)- y. Using the chain rule once more 
we find 


gw) =X D(ED,Slewlvs) = YY Duflwyy, = yHUMwY, 


Hence g"(c) = yH(a + cy)y', so Equation (9.36) becomes (9.34). 
To prove (9.35) we define E,(a, y) by the equation 


1 
(9.37) lly|I° E.(@, y) = 5 AH + oy) — H(a)}y' if yO, 
and let E,(a, O) = 0. Then Equation (9.34) takes the form 
1 
fat+ty)-f@M=VE@-y+ 5 vH@y + |lyll’ E.(a, y). 


To complete the proof we need to show that F,(a, y) ~0asy—> O. 
From (9.37) we find that 


S> {D.,f(a + cy) — Dy,f@} yyy, 


i=1 j=1 


lll" |E.(a, y)| = = 


<!> S Dy fla + ey) — Dy f(@| lye. 


Sa ara 
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Dividing by ||y||? we obtain the inequality 


1 n n 
Ea WS 5 > > [Disfla + ey) — Di fl 


i=1 j=1 


for y # O. Since each second-order partial derivative D;,f is continuous at a, we have 
Dj, f(a + cy) > Di; f(@ as y > O, so E,(a, y) > 0 as y—> O. This completes the proof. 


9.11. The nature of a stationary point determined by the eigenvalues of the Hessian matrix 


At a stationary point we have Vf(a) = O, so the Taylor formula in Equation (9.35) 
becomes 


f(a + y) —f(@ = tyH(a)y' + |lyll? E.(a, y). 


Since the error term ||y||? E,(a, y) tends to zero faster than ||y||?, it seems reasonable to 
expect that for small y the algebraic sign of f(a + y) — f(a) is the same as that of the 
quadratic form yH(a)y'; hence the nature of the stationary point should be determined by 
the algebraic sign of the quadratic form. This section is devoted to a proof of this fact. 

First we give a connection between the algebraic sign of a quadratic form and its eigen- 
values. 


THEOREM 9.5. Let A = [a,;] be ann x n real symmetric matrix, and let 


Oy) = yay’ = XD asiys- 
az J= 
Then we have: 


(a) O(y) > 0 for all y ¥ O if and only if all the eigenvalues of A are positive. 
(b) O(y) < 0 for all y ¥ O if and only if all the eigenvalues of A are negative. 


Note: Incase (a), the quadratic form is called positive definite; in case (b) it is called 
negative definite. 


Proof. According to Theorem 5.11 there is an orthogonal matrix C that reduces the 
quadratic form yAy’ to a diagonal form. That is 


(9.38) O(y) = yAy’ = p? 1;x% 


where x = (x,,...,X,) is the row matrix x = yC, and /,,..., A, are the eigenvalues of 
A. The eigenvalues are real since A is symmetric. 

If all the eigenvalues are positive, Equation (9.38) shows that Q(y) > 0 whenever 
x # O. But since x = yC we have y = xC 1, so x $ Oif and only if y # O. Therefore 
O(y) > 0 for all y ¥ O. 

Conversely, if Q(y) > 0 for all y # O we can choose y so that x = yC is the kth co- 
ordinate vector e,. For this y, Equation (9.38) gives us Q(y) = A,, so each A, > 0. This 
proves part (a). The proof of (b) is entirely analogous. 
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The next theorem describes the nature of a stationary point in terms of the algebraic 
sign of the quadratic form yH(a)y’. 


THEOREM 9.6. Let f be a scalar field with continuous second-order partial derivatives 
D;,;f in an n-ball B(a), and let H(a) denote the Hessian matrix at a stationary point a. Then 
we have: 

(a) If all the eigenvalues of H(a) are positive, f has a relative minimum at a. 

(b) [fall the eigenvalues of H(a) are negative, f has a relative maximum at a. 

(c) If H(a) has both positive and negative eigenvalues, then f has a saddle point at a. 


Proof. Let Q(y) = yH(a@)y'. The Taylor formula gives us 


(9.39) f(a + y) —f(@) = 4Q(y) + lly? £2(@, y), 


where E,(a, y) > 0 as y— O. We will prove that there is a positive number r such that, if 
0 < |ly|l <r, the algebraic sign of f(a + y) — f(a) is the same as that of Q(y). 

Assume first that all the eigenvalues 2,, ..., A, of H(a) are positive. Let h be the 
smallest eigenvalue. If u < h, the nm numbers 


A,—Uu,...,4,—U 


are also positive. These numbers are the eigenvalues of the real symmetric matrix H(a) — 
ul, where J is the n Xn identity matrix. By Theorem 9.5, the quadratic form 
y[H(a) — ull]y' is positive definite, and hence y[H(a) — ul]ly' > 0 for all y 4 O. There- 
fore 


yH(a)y' > y(uly* = u |ly|l? 


for all realu << h. Taking u = 3h we obtain the inequality 


O(y) > th llyll? 


for ally # O. Since E,(a, y)—> O as y — O, there is a positive number r such that 
|E.(a, y)| < +h whenever 0 < ||y|| <r. For such y we have 


0 < llyll? [Ee(a, »)| < 2h llyll? < 200), 


and Taylor’s formula (9.39) shows that 


f(at+y) — f(a) > 22) — lly? |E2(@, y)| > 0. 


Therefore f has a relative minimum at a, which proves part (a). To prove (b) we can use a 
similar argument, or simply apply part (a) to —f. 

To prove (c), let A, and A, be two eigenvalues of H(a) of opposite signs. Let h = 
min {|A,|, |a)|}. Then for each real u satisfying —h < u < h the numbers 


Ay — u and Ag — U 
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are eigenvalues of opposite sign for the matrix H(a) — ul. Therefore, if ue (—h, h), the 
quadratic form y[H(a) — ul]y* takes both positive and negative values in every neighbor- 
hood of y = O. Choose r > 0 as above so that |E,(a, y)| < +h whenever 0 < |ly|| <r. 
Then, arguing as above, we see that for such y the algebraic sign of f(a + y) — f(a) is the 
same as that of Q(y). Since both positive and negative values occur as y—> O, f has a 
saddle point at a. This completes the proof. 


Note: If all the eigenvalues of H(a) are zero, Theorem 9.6 gives no information 
concerning the stationary point. Tests involving higher order derivatives can be used to 
treat such examples, but we shall not discuss them here. 


9.12 Second-derivative test for extrema of functions of two variables 


In the case n = 2 the nature of the stationary point can also be determined by the 
algebraic sign of the second derivative D, , f(a) and the determinant of the Hessian matrix. 


THEOREM 9.7. Let a be a stationary point of a scalar field f(x, X2) with continuous second- 
order partial derivatives in a 2-ball B(a). Let 


A=D,,f(@, B= D, f(a), C= Dyof(a), 
and let 


A B 
A = det H(a) = det = AC — B?, 
BC 


Then we have: 
(a) IfA <0, fhas a saddle point at a. 
(b) fA > 0 and A > 0, fhas a relative minimum at a. 
(c) IfA>Oand A <0, fhas a relative maximum at a. 
(d) If A = 0, the test is inconclusive. 


Proof. In this case the characteristic equation det [AJ — H(a)] = 0 is a quadratic 
equation, 


The eigenvalues A,, 4, are related to the coefficients by the equations 
A +tA,=AtC, Asn = A. 


If A < 0 the eigenvalues have opposite signs, so f has a saddle point at a, which proves (a). 
If A > 0, the eigenvalues have the same sign. In this case AC > B? > 0, so A and C have 
the same sign. This sign must be that of A, and A, since A + C =A, + A,. This proves 
(b) and (c). 

To prove (d) we refer to Examples 4 and 5 of Section 9.9. In both these examples we have 
A = 0 at the origin. In Example 4 the origin is a saddle point, and in Example 5 it is a 
relative minimum. 
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Even when Theorem 9.7 is applicable it may not be the simplest way to determine the 
nature of a stationary point. For example, when f(x, y) = e/9%, where g(x, y) = 
x? + 2 + cos? y — 2 os y, the test is applicable, but the computations are lengthy. In this 
case we may express g(x, y) as asum of squares by writing g(x, y) = 1 + x? + (1 — cos y)?. 
We see at once that fhas relative maxima at the points at which x? = 0 and (1 — cos y)? = 
QO. These are the points (0, 27), when n is any integer. 


9.13. Exercises 


In Exercises 1 through 15, locate and classify the stationary points (if any) of the surfaces having 
the Cartesian equations given. 


1. z =x? + (y — 1). 7. z =x? — 3xy? + y?®. 
2.z=x? —(y —- 1). 8. z = x*y3(6 —x —y). 
3.z=14+x?-y?. 9.z =x? + y? — 3xy. 
4.z=(x-—yt1). 10. z = sinxcoshy. 

5. z = 2x? — xy — 3y? — 3x +7y. 11. z = e?+8u(8x? — 6xy + 3y?). 
6.zZ=x?—xy+y?—2x+y. 12. z = (5x + Ty — 25)e (e*tevty") | 
13. z = sinxsinysin(x + y), O<x<e7,0Sy<r. 


14.z =x —2y +log /x* + y? + 3arctan~, x >0. 
15. z= (x? + yre—(aPty?) | 


16. Let f(x, y) = 3x* — 4x?y + y?. Show that on every line y = mx the function has a minimum 
at (0, 0), but that there is no relative minimum in any two-dimensional neighborhood of the 
origin. Make a sketch indicating the set of points (x, y) at which f(x, y) > 0 and the set at 
which f(x, y) < 0. 

17. Let f(x, y) = 3 — x)3 — yx + y— 3). 

(a) Make a sketch indicating the set of points (x, y) at which f(x, y) > 0. 

(b) Find all points (x, y) in the plane at which D, f(x, y) = Dof(x, y) =0. [Hint: D, f(x, y) 
has (3 — y) as a factor.] 

(c) Which of the stationary points are relative maxima? Which are relative minima? Which 
are neither? Give reasons for your answers. 

(d) Does f have an absolute minimum or an absolute maximum on the whole plane? Give 
reasons for your answers. 

18. Determine all the relative and absolute extreme values and the saddle points for the function 
f(x, y) = xy(1 — x? — y*) on the squareO <x <1,0<y<l. 

19. Determine constants a and 5 such that the integral 


ib {ax +b — f(x)}* dx 


will be as small as possible if (a) f(x) = x?; (b) f(x) = (2? + 1)?. 
20. Let f(x, y) = Ax® + 2Bxy + Cy? + 2Dx + 2Ey + F, where A > 0 and B® < AC. 
(a) Prove that a point (x,, y,) exists at which fhas a minimum. [Hint: Transform the quad- 
ratic part to a sum of squares. ] 
(b) Prove that f(x, y,) = Dx, + Ey, + Fat this minimum. 
(c) Show that 
A B D 


1 
I) = Far Re B C E}\. 


D E F 
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21; 


ae: 


23. 


24. 


20: 


Method of least squares. Given n distinct numbers x,,..., x, and further numbers y,,..., 
yn (not necessarily distinct), it is generally impossible to find a straight line f(x) = ax +b 
which passes through all the points (x;, y;), that is, such that f(x;) = y,; foreach i. However, 
we can try a linear function which makes the “total square error’”’ 


E(a, b) = p) Lf) — yP 


a minimum. Determine values of a and b which do this. 
Extend the method of least squares to 3-space. That is, find a linear function f(x, y) = ax + 
by + c which minimizes the total square error 


E(a, b,c) = ¥ Ufai.y) — 22, 
7i=1 


where (x;, y;) are n given distinct points and z,,..., Z, are nm given real numbers. 
Let z,,..., 2, be n distinct points in m-space. If x E R™, define 


fe) = ¥ Ix — al. 
k=1 


n 


1 
Prove that fhas a minimum at the point a = is > Z;, (the centroid). 


k=1 
Let a be a stationary point of a scalar field f with continuous second-order partial derivatives 
in an n-ball B(a). Prove that f has a saddle point at a if at least two of the diagonal entries 
of the Hessian matrix H(a) have opposite signs. 
Verify that the scalar field f(x, y, z) = x* + y* + z* — 4xyz has a stationary point at (1, 1, 1), 
and determine the nature of this stationary point by computing the eigenvalues of its Hessian 
matrix. 


9.14 Extrema with constraints. Lagrange’s multipliers 


We begin this section with two examples of extremum problems with constraints. 


EXAMPLE 1. Given a surface S not passing through the origin, determine those points of 


S which are nearest to the origin. 


EXAMPLE 2. If f(x, y, z) denotes the temperature at (x, y, z), determine the maximum 


and minimum values of the temperature on a given curve C in 3-space. 


Both these examples are special cases of the following general problem: Determine the 


extreme values of a scalar field f(x) when x is restricted to lie in a given subset of the domain 


of f. 


In Example 1 the scalar field to be minimized is the distance function, 


S(x,y; Z) = (x? + y + zy ; 
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the constraining subset is the given surface S. In Example 2 the constraining subset is the 
given curve C. 

Constrained extremum problems are often very difficult; no general method is known 
for attacking them in their fullest generality. Special methods are available when the 
constraining subset has a fairly simple structure, for instance, if it is a surface as in Example 
1, or a curve as in Example 2. This section discusses the method of Lagrange’s multipliers 
for solving such problems. First we describe the method in its general form, and then we 
give geometric arguments to show why it works in the two examples mentioned above. 


The method of Lagrange’s multipliers. If a scalar field f(x,,...,x,) has a relative 
extremum when it is subject to m constraints, say 


(9.40) £:(x,,...,X,) = 0, Soe. Mel Missaesx,) = 0; 
where m <n, then there exist m scalars 4,,..., A,, Such that 

(9.41) Vf = A, Ver tee bAn VEm 

at each extremum point. 


To determine the extremum points in practice we consider the system of n + m equations 
obtained by taking the m constraint equations in (9.40) along with the n scalar equations 
determined by the vector relation (9.41). These equations are to be solved (if possible) for 
the n + m unknowns x,,..., x, and 4,,...,4,,. The points (x,,...,x,) at which 
relative extrema occur are found among the solutions to these equations. 

The scalars 4,,..., 4,, which are introduced to help us solve this type of problem are 
called Lagrange’s multipliers. One multiplier is introduced for each constraint. The scalar 
field f and the constraint functions g,, ..., 2m are assumed to be differentiable. The 
method is valid if the number of constraints, m, is less than the number of variables, n, and 
if not all the Jacobian determinants of the constraint functions with respect to m of the 
variables x,,...,X, are zero at the extreme value in question. The proof of the validity of 
the method is an important result in advanced calculus and will not be discussed here. 
(See Chapter 7 of the author’s Mathematical Analysis, Addison-Wesley, Reading, Mass., 
1957.) Instead we give geometric arguments to show why the method works in the two 
examples described at the beginning of this section. 


Geometric solution of Example 1. We wish to determine those points on a given surface 
S which are nearest to the origin. A point (x, y, z) in 3-space lies at a distance r from the 
origin if and only if it lies on the sphere 


x2 y? + 27 = 7? 


This sphere is a level surface of the function f(x, y, z) = (x? + y? + z?)* which is being 
minimized. If we start with r = 0 and let r increase until the corresponding level surface 
first touches the given surface S, each point of contact will be a point of S nearest to the 
origin. 
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To determine the coordinates of the contact points we assume that S is described by a 
Cartesian equation g(x, y, z) = 0. If Shas a tangent plane at a point of contact, this plane 
must also be tangent to the contacting level surface. Therefore the gradient vector of the 
surface g(x, y, z) = 0 must be parallel to the gradient vector of the contacting level surface 
T(x, y, Z) =r. Hence there is a constant A such that 


Vf(x, y, z) = A Ve(x, y, Z) 


at each contact point. This is the vector equation (9.41) provided by Lagrange’s method 
when there is one constraint. 


Geometric solution to Example 2. We seek the extreme values of a temperature function 
f(x, y, Z) on a given curve C. If we regard the curve C as the intersection of two surfaces, 
say 

g(x, y,z) =0 and =e g(x, y,z) = 0, 


we have an extremum problem with two constraints. The two gradient vectors Vg, and 
Vg. are normals to these surfaces, hence they are also normal to C, the curve of inter- 
section. (See Figure 9.8.) We show next that the gradient vector Vf of the temperature 


FIGURE 9.8 The vectors Vg,, Vg., and Vf FIGURE 9.9 The gradient vector Vf lies in a 
shown lying in the same plane. plane normal to C. 


function is also normal to C at each relative extremum on C. This implies that Vflies in the 
same plane as Vg, and Vg.; hence if Vg, and Vg, are independent we can express Vf as a 
linear combination of Vg, and Vg., say 


Vf = A,Ve, + AVeo- 


This is the vector equation (9.41) provided by Lagrange’s method when there are two 
constraints. 
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To show that Vf is normal to C at an extremum we imagine C as being described by a 
vector-valued function a(t), where ¢ varies over an interval [a,b]. On the curve C the 
temperature becomes a function of f, say y(t) = f[a(r)]. If y has a relative extremum at an 
interior point ¢, of [a, b] we must have g’(t,) = 0. On the other hand, the chain rule tells 
us that y’(t) is given by the dot product 


g(t) = Vfla(t)]- a(t). 


This dot product is zero at ¢,, hence Vf is perpendicular to a’(t,). But a’(1,) is tangent to C, 
so Vf[a(t,)] lies in the plane normal to C, as shown in Figure 9.9. 


The two gradient vectors Vg, and Vg, are independent if and only if their cross product is 
nonzero. The cross product is given by 


i J k 
Vex Veg= Og, Og, 0g, ” O(g;, 22). , Ag, 82) O(81, Bo) k. 
Ox Oy az O(y, Z) O(z, x) a(x, y) 


O82 O82 8s 
Ox Oy @z 


Therefore, independence of Vg, and Vg. means that not all three of the Jacobian deter- 
minants on the right are zero. As remarked earlier, Lagrange’s method is applicable 
whenever this condition is satisfied. 


If Vg, and Vg, are dependent the method may fail. For example, suppose we try to apply 
Lagrange’s method to find the extreme values of f(x, y, z) = x® + y? on the curve of 
intersection of the two surfaces g,(x, y, z) = 0 and g.(x, y, z) = 0, where g,(x, y, z) = 2 
and go(x, y, z) =z? — (y — 1)*. The two surfaces, a plane and a cylinder, intersect along 
the straight line C shown in Figure 9.10. The problem obviously has a solution, because 


FiGure 9.10 An example where Lagrange’s method is not applicable. 
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I(x, y, z) represents the distance of the point (x, y, z) from the z-axis and this distance is a 
minimum on C when the point is at (0, 1, 0). However, at this point the gradient vectors 
are Vg, = K, Vg. = O, and Vf = 2f, and it is clear that there are no scalars A, and A, that 
satisfy Equation (9.41). 


9.15 Exercises 


1. Find the extreme values of z = xy subject to the condition x + y = 1. 
2. Find the maximum and minimum distances from the origin to the curve 5x? + 6xy + Sy? = 8. 
3. Assume a and 6 are fixed positive numbers. 
(a) Find the extreme values of z = x/a + y/b subject to the condition x* + y2 =1. 
(b) Find the extreme values of z = x? + y® subject to the condition x/a + y/b = 1. 
In each case, interpret the problem geometrically. 
4. Find the extreme values of z = cos? x + cos? y subject to the side condition x — y = 7/4. 
5. Find the extreme values of the scalar field f(x, y, z) = x — 2y + 2z on the sphere x? + y® + 
zz=1. 
6. Find the points of the surface z? — xy = 1 nearest to the origin. 
7. Find the shortest distance from the point (1, 0) to the parabola y® = 4x. 
8. Find the points on the curve of intersection of the two surfaces 


x2 


—xy+ty-2=1 and x+y?=1 
which are nearest to the origin. 
9. If a, b, and c are positive numbers, find the maximum value of f(x, y, z) = x*y’z° subject to 
the side conditionx +y+z=1. 
10. Find the minimum volume bounded by the planes x = 0, y = 0, z = 0, and a plane which 
is tangent to the ellipsoid 
2 z2 


x? 
x acer ieee, 


a Be 


at a point in the octantx >0,y >0,z>0. 

11. Find the maximum of log x + log y + 3 log zon that portion of the sphere x? + y? + z? = 5r? 
where x > 0, y >0,2z>0. Use the result to prove that for real positive numbers a, b, c 
we have 

at+tb+c~y 
abe? < (272 *) 


12. Given the conic section Ax? + 2Bxy + Cy? = 1, where A > 0 and B? < AC. Let mand M 
denote the distances from the origin to the nearest and furthest points of the conic. Show that 


weet +C+J/(A — C)? + 4B? 

7 2(AC — B?) 
and find a companion formula for mm. 

13. Use the method of Lagrange’s multipliers to find the greatest and least distances of a point 
on the ellipse x? + 4y? = 4 from the straight line x + y = 4. 

14. The cross section of a trough is an isosceles trapezoid. If the trough is made by bending up 
the sides of a strip of metal c inches wide, what should be the angle of inclination of the sides 
and the width across the bottom if the cross-sectional area is to be a maximum? 
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9.16 The extreme-value theorem for continuous scalar fields 


The extreme-value theorem for real-valued functions continuous on a closed and bounded 
interval can be extended to scalar fields. We consider scalar fields continuous on a closed 
n-dimensional interval. Such an interval is defined as the Cartesian product of n one- 
dimensional closed intervals. If a = (a,,...,a,) and b = (b,,...,5,) we write 


[a, 5] = [a,, 5,] x "°° kK [a,,5,] = ower ° ,Xy)| x1 [a,,5,],. oe prin SS [a,, 5,]}. 


For example, when n = 2 the Cartesian product [a, b] is a rectangle. 

The proof of the extreme-value theorem parallels the proof given in Volume I for the 
1-dimensional case. First we prove that continuity of fimplies boundedness, then we prove 
that f actually attains its maximum and minimum values somewhere in [a, 5]. 


THEOREM 9.8. BOUNDEDNESS THEOREM FOR CONTINUOUS SCALAR FIELDS. Jf f is a scalar 
field continuous at each point of a closed interval |a, b] in R”, then f is bounded on {a, b]. 
That is, there is a number C > 0 such that | f(x)| < C for all x in [a, b). 


Proof. We argue by contradiction, using the method of successive bisection. Figure 
9.11 illustrates the method for the case n = 2. 
Assume f is unbounded on [a, 6]. Let J") = [a, b] and let 7) = [a,, b,], so that 


y™ = yo KX je 


Bisect each one-dimensional interval J,’ to form two subintervals, a left half J) and a 
right half 7/2}. Now consider all possible Cartesian products of the form 


fe ror sed 


N,5n 9 


FIGURE 9.11 Illustrating the method of successive bisection in the plane. 
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where each /; = | or 2. There are exactly 2" such products. Each product is an n-dimen- 
sional subinterval of [a, 5], and their union is equal to [a, b]. The function fis unbounded 


in at least one of these subintervals (if it were bounded in each of them it would also be 
bounded on [a, b]). One of these we denote by /‘?) which we express as 


] = 1 K+ & jaar 


where each /'?) is one of the one-dimensional subintervals of /)!), of length $(b, — a,). 
We now proceed with /'?) as we did with /™, bisecting each one-dimensional component 
interval /,?’ and arriving at an n-dimensional interval /) in which fis unbounded. We 
continue the process, obtaining an infinite set of n-dimensional intervals 
PON Ng aa with mt) © Jim, 
in each of which fis unbounded. The mth interval J’) can be expressed in the form 


p™ = io K+ & Lo. 


Since each one-dimensional interval /;”) is obtained by m — 1 successive bisections of 


[a,, b,], if we write 1°” = [ay”, b)”] we have 
” ” b,— a 
(9.42) le a aap: for k=1,2,...,n. 


For each fixed k, the supremum of all left endpoints a." (m = 1, 2,...) must therefore be 
equal to the infimum of all right endpoints 6.” (m = 1, 2,...), and their common value 
we denote by ¢,,. The pointt = (t,,...,7,,) lies in [a, b]. By continuity of fat ¢ there is an 
n-ball B(t; r) in which we have 


f(x) —f(O| < 1 for all x in B(t; r) A [a, 5]. 
This inequality implies 
f(x) << 1 + [fl for all x in B(t;r) CO [a, 5], 


so fis bounded on the set B(t;r) A [a,b]. But this set contains the entire interval /‘” 
when mis large enough so that each of the n numbers in (9.42) is less than rfs/n. Therefore 
for such m the function fis bounded on /‘™, contradicting the fact that fis unbounded on 
I‘™), This contradiction completes the proof. 


If f is bounded on [a, b], the set of all function values f(x) is a set of real numbers 


bounded above and below. Therefore this set has a supremum and an infimum which we 
denote by sup f and inf/, respectively. That is, we write 


sup f = sup {f(x)|xe¢[a,4]}, inf f = inf {f(x)| x € [a, d]}. 
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Now we prove that a continuous function takes on both values inf fand sup f somewhere 
in [a, 5). 


THEOREM 9.9 EXTREME-VALUE THEOREM FOR CONTINUOUS SCALAR FIELDS. If f is con- 
tinuous on a closed interval [a, b) in R", then there exist points c and d in |{a, b) such that 


f(c)=supf and f(d) =inf/f. 


Proof. It suffices to prove that f attains its supremum in [a,b]. The result for the 
infimum then follows as a consequence because the infimum of fis the supremum of —f. 

Let M = sup/. We shall assume that there is no x in [a, b] for which f(x) = M and 
obtain a contradiction. Let g(x) = M — f(x). Then g(x) > 0 for all x in [a, b] so the 
reciprocal |/g is continuous on [a, 5]. By the boundedness theorem, 1/g is bounded on 
[a, b], say 1/g(x) < C for all x in [a, b], where C > 0. This implies M — f(x) > 1/C, so 
that f(x) << M — 1/C for all x in [a, 5]. This contradicts the fact that A/ is the least upper 
bound of fon [a, b]. Hence f(x) = M for at least one x in [a, ]. 


9.17 The small-span theorem for continuous scalar fields (uniform continuity) 


Let f be continuous on a bounded closed interval [a, 6] in R”, and let M(f) and m(/) 
denote, respectively, the maximum and minimum values of fon [a, b]. The difference 


M(f) — m(f). 


is called the span of f on [a,b]. As in the one-dimensional case we have a small-span 
theorem for continuous functions which tells us that the interval [a, b] can be partitioned so 
that the span of fin each subinterval is arbitrarily small. 

Write [a, b] = [a,,5,] X --- X [a,,5,], and let P, be a partition of the interval 
[a,, 5,]. That is, P, is a set of points 


Pye {Xp Kiser g ia Xe} 
such that a4, = x» CX, °° SX SX, = D,. The Cartesian product 
P oe > ie eee P,. 


is called a partition of the interval [a, b]. The small-span theorem, also called the theorem 
on uniform continuity, now takes the following form. 


THEOREM 9.10. Let f be a scalar field continuous on a closed interval |a, b| in R". Then 
for every « > 0 there is a partition of [a, b] into a finite number of subintervals such that the 
span of f in every subinterval is less than e. 


Proof. The proof is entirely analogous to the one-dimensional case so we only outline 
the principal steps. We argue by contradiction, using the method of successive bisection. 
We assume the theorem is false; that 1s, we assume that for some e, the interval [a, 5] 
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cannot be partitioned into a finite number of subintervals in each of which the span of fis 
less than €,g. By successive bisection we obtain an infinite set of subintervals J™, 7, ..., 
in each of which the span of fis at least eg. By considering the least upper bound of the 
leftmost endpoints of the component intervals of J, 7, ... we obtain a point ¢ in [a, 5] 
lying in all these intervals. By continuity of fat ¢ there is an n-ball B(t; r) such that the span 
of fis less than $e) in B(t; r) A [a, 6]. But, when m is sufficiently large, the interval J‘ 
lies in the set B(t; r) A [a, b], so the span of fis no larger than $e, in /'™, contradicting the 
fact that the span of fis at least €) in J‘™. 


10 


LINE INTEGRALS 


10.1 Introduction 


In Volume I we discussed the integral f? f(x) dx, first for real-valued functions defined 
and bounded on finite intervals, and then for unbounded functions and infinite intervals. 
The concept was later extended to vector-valued functions and, in Chapter 7 of Volume IT, 
to matrix functions. 

This chapter extends the notion of integral in another direction. The interval [a, 5] is 
replaced by a curve in n-space described by a vector-valued function a, and the integrand 
is a vector field f defined and bounded on this curve. The resulting integral is called a /ine 
integral, a curvilinear integral, or a contour integral, and is denoted by ff: da or by some 
similar symbol. The dot is used purposely to suggest an inner product of two vectors. The 
curve is called a path of integration. 

Line integrals are of fundamental importance in both pure and applied mathematics. 
They occur in connection with work, potential energy, heat flow, change in entropy, 
circulation ofa fluid, and other physical situations in which the behavior of a vector or 
scalar field 1s studied along a curve. 


10.2 Paths and line integrals 


Before defining line integrals we recall the definition of curve given in Volume I. Let @ 
be a vector-valued function defined on a finite closed interval J = [a, b]. As truns through 
J, the function values a(t) trace out a set of points in n-space called the graph of the function. 
If a is continuous on J the graph is called a curve; more specifically, the curve described 
by a. 

In our study of curves in Volume I we found that different functions can trace out the 
same curve in different ways, for example, in different directions or with different velocities. 
In the study of line integrals we are concerned not only with the set of points on a curve but 
with the actual manner in which the curve is traced out, that 1s, with the function @ itself. 
Such a function will be called a continuous path. 


DEFINITION. Let J = [a, b] be a finite closed interval in R'. A function a: J — R" which 
is continuous on J is called a continuous path in n-space. The path is called smooth if the 
derivative a exists and is continuous in the open interval (a, b). The path is called piecewise 
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smooth if the interval [a, b] can be partitioned into a finite number of subintervals in each of 
which the path is smooth. 


Figure 10.1 shows the graph of a piecewise smooth path. In this example the curve has a 
tangent line at all but a finite number of its points. These exceptional points subdivide the 
curve into arcs, along each of which the tangent line turns continuously. 


Ficure 10.1. The graph of a piecewise smooth path in the plane. 


DEFINITION OF LINE INTEGRAL. Let «@ be a piecewise smooth path in n-space defined on an 
interval [a, b}, and let f be a vector field defined and bounded on the graph of a. The line 
integral of f along « is denoted by the symbol | f: da and is defined by the equation 


b 
(10.1) [ f-do = |” flay] a's at, 
whenever the integral on the right exists, either as a proper or improper integral. 


Note: Inmost examples that occur in practice the dot product f[a(r)]- a’ (t)is bounded 
on [a, b] and continuous except possibly at a finite number of points, in which case the 
integral exists as a proper integral. 


10.3. Other notations for line integrals 


If C denotes the graph of a, the line integral | f- da is also written as fo f- da and is 
called the integral of f along C. 

If a = a(a) and b = a(b) denote the end points of C, the line integral is sometimes 
written as f® for as [2 f- da and is called the line integral of f from a to b along a. When 
the notation f® fis used it should be kept in mind that the integral depends not only on the 
end points a and b but also on the path @ joining them. 

When a = b the path is said to be closed. The symbol ¢ is often used to indicate integra- 
tion along a closed path. 

When f and @ are expressed in terms of their components, say 


I=] Sigcvesty) and a= (a%,,...,%,), 


the integral on the right of (10.1) becomes a sum of integrals, 


& PP falatnouco at. 


In this case the line integral is also written as { f, da, +--+ +f, day. 
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In the two-dimensional case the path @ is usually described by a pair of parametric 
equations, 


x=oy(t), y= a(t), 


and the line integral J, f- dais written as fo fy dx + fody, oras fo f(x, y)dx + falx,y) dy. 
In the three-dimensional case we use three parametric equations, 


x=a(t), yoalt), z= a(t) 


and write the line integral Jo f: da as fo f, dx + fody +f, dz, or as 
[Als y. 2) dx + falas y, 2) dy + folx, y, 2) dz. 
EXAMPLE. Let f be a two-dimensional vector field given by 


f(x,y) = Jy it (08 + y)j 


for all (x, y) with y > 0. Calculate the line integral of ffrom (0, 0) to (1, 1) along each of 
the following paths: 

(a) the line with parametric equations x =t,y=?t,0<?t<1; 

(b) the path with parametric equationsx =?,y=f,0<t<l. 


Solution. For the path in part (a) we take a(t) = “+ 4. Then a(t) =i+ f and 
fla(t)] = vire (t?+ t)j. Therefore the dot product of f[a(t)] and a’(t) is equal to 
s/t + t? + t and we find 

(1,1) 17 


feda= | (Witt +nar= 
0 12 


(0,0) 


For the path in part (b) we take a(t) = 784+ fF. Then a(t) = 208+ 343 and 
fla()] = 147 + (t® + )j. Therefore 


fla(t)] > a(t) = 21% + 34° + 39°, 
NTO) 


(1,1) 1 59 
| f-da= | (27% + 3°+4+ 3P)dt= —. 
0 42 


(0,0) 
This example shows that the integral from one point to another may depend on the path 
joining the two points. 


Now let us carry out the calculation for part (b) once more, using the same curve but 
with a different parametric representation. The same curve can be described by the 
function 


Bir) =n +1%j, where 0O<1t<1. 
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This leads to the relation 
FIBO]: BOD = (4i + (08 + )j)- EAE) = 4 + Bt + Gt, 


the integral of which from 0 to 1 is 59/42, as before. This calculation illustrates that the 
value of the integral is independent of the parametric representation used to describe the 
curve. This is a general property of line integrals which is proved in the next section. 


10.4 Basic properties of line integrals 


Since line integrals are defined in terms of ordinary integrals, it is not surprising to find 
that they share many of the properties of ordinary integrals. For example, they have a 
linearity property with respect to the integrand, 


| aft bg)-da=alf-da+b]g-da, 


and an additive property with respect to the path of integration: 


I. f-da= I. f-da+ Io. f- da, 


where the two curves C, and C, make up the curve C. That is, Cis described by a function 
a defined on an interval [a, 5], and the curves C, and C, are those traced out by a(t) as ¢ 
varies over subintervals [a,c] and [c, b], respectively, for some c satisfying a<ec<b. 
The proofs of these properties follow immediately from the definition of the line integral; 
they are left as exercises for the reader. 

Next we examine the behavior of line integrals under a change of parameter. Let a be a 
continuous path defined on an interval [a, b], let u be a real-valued function that is differen- 
tiable, with uw’ never zero on an interval [c, d], and such that the range of u is [a, b]. Then 
the function B defined on [c, d] by the equation 


B(t) = a[u(t)] 


is a continuous path having the same graph as a. Two paths @ and 8 so related are called 
equivalent. They are said to provide different parametric representations of the same curve. 
The function wu is said to define a change of parameter. 

Let C denote the common graph of two equivalent paths a and B. If the derivative of u 
is always positive on [c, d] the function uw is increasing and we say that the two paths a and B 
trace out C in the same direction. If the derivative of u is always negative we say that a 
and B trace out C in opposite directions. In the first case the function wu is said to be 
orientation-preserving; in the second case u is said to be orientation-reversing. An example 
is shown in Figure 10.2. 

The next theorem shows that a line integral remains unchanged under a change of param- 
eter that preserves orientation; it reverses its sign if the change of parameter reverses 
orientation. We assume both intergals { f- da and { f- dB exist. 
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(b) 


FiGure 10.2 A change of parameter defined by u = A(t). In (a), the function h 
preserves orientation. In (b), the function h reverses the orientation. 


THEOREM 10.1. BEHAVIOR OF A LINE INTEGRAL UNDER A CHANGE OF PARAMETER. Let 
a and 8 be equivalent piecewise smooth paths. Then we have 


|, fda =| fap 


if a and B trace out C in the same direction; and 


|, fda=—| f- ap 
if « and B trace out C in opposite directions. 


Proof. It suffices to prove the theorem for smooth paths; then we invoke 
the additive property with respect to the path of integration to deduce the result for piece- 
wise smooth paths. 

The proof is a simple application of the chain rule. The paths @ and B are related by an 
equation of the form B(t) = a[u(t)], where wu is defined on an interval [c, d] and @ is defined 
on an interval [a, b]. From the chain rule we have 


BY(t) = a [u(t)u'(t). 


Therefore we find 


\, fd = [F180 “BW dt = [" fata) “a [u(t)]u'(t) dt. 


In the last integral we introduce the substitution v = u(t), dv = u'(t) dt to obtain 


[£48 = J“° slate) av) dv = +] fla(v))-a'(v) do = $f da, 


U 
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where the + sign is used if a = u(c) and b = u(d), and the — sign is used if a = u(d) and 
b =u(c). The first case occurs if a and B trace out C in the same direction, the second if 
they trace out C in opposite directions. 


10.5 Exercises 


In each of Exercises 1 through 8 calculate the line integral of the vector field f along the path 
described. 


1. f(x, y) = (x? — 2xy)i + (y? — 2xy)j, from (—1, 1) to (1, 1) along the parabola y = x?. 

2. f(x, y) = (2a — y)i + xj, along the path described by a(t) = a(t — sint)i + a(1 — cos ry, 
0O<t<27. 

3. f(x, y, Zz) = (? — 2*)i + 2yzj — x*k, along the path described by a(t) = si +277 + 2°k, 
O<r<l. 

4. f(x, y) = (7 + y®)i + (x? — y®)j, from (0, 0) to (2, 0) along the curve y = 1 — |1 — x|. 

5. f(x, y) = (x + y)i + (x — y)j, once around the ellipse b?x? + a?y? = ab? in a counterclock- 
wise direction. 

6. f(x, y, Z) = 2xyi + (x? + z)j + yk, from (1, 0, 2) to (3, 4, 1) along a line segment. 

7. f(x, y, Z) = xi + yi + (xz — y)k, from (0, 0, 0) to (1, 2, 4) along a line segment. 

8. f(x, y,z) = xi + yi + (xz — y)k, along the path described by a(t) =¢7i + 247 + 4¢°k, 
0O<r<l. 


In each of Exercises 9 through 12, compute the value of the given line integral. 


9. fo (x? — 2xy) dx + (y? — 2xy) dy, where C is a path from (—2, 4) to (1, 1) along the parabola 


a ae 
> (x + y)dx —(x -—y)d 
10. | Cag 1 lex eee 2 , where C is the circle x? + y? = qa’, traversed once in a counter- 
Cc x? + y? 
clockwise direction. 
dx + dy : ; 
11. {. xl + pl , where C is the square with vertices (1, 0), (0, 1), (—1, 0), and (0, —1), traversed 


once in a counterclockwise direction. 

12. Jo ydx +zdy +x dz, where 
(a) Cis the curve of intersection of the two surfaces x + y = 2and x? + y? +z? =2(x + y). 
The curve is to be traversed once in a direction that appears clockwise when viewed from the 
origin. 
(b) C is the intersection of the two surfaces z = xy and x? + y* = 1, traversed once in a 
direction that appears counterclockwise when viewed from high above the xy-plane. 


10.6 The concept of work as a line intregal 


Consider a particle which moves along a curve under the action of a force field f. If the 
curve is the graph of a piecewise smooth path a, the work done by fis defined to be the line 
integral | f- da. The following examples illustrate some fundamental properties of work. 


EXAMPLE 1. Work done by a constant force. If fis a constant force, say f= c, it can be 
shown that the work done by fin moving a particle from a point a to a point b along any 
piecewise smooth path joining a and b is ec: (b — a), the dot product of the force and the 
displacement vector b — a. We shall prove this in a special case. 

We let a = (a,,...,«,) be a path joining a and 6, say a(a) = a and a(b) = Bb, and we 
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write ¢ = (c,,...,¢,). Assume @’ is continuous on [a,b]. Then the work done by f is 
equal to 


[fda = Seq J a(t) dt = ¥ egley(b) — a4(a)] = €  [a(b) — a(a)] = €- (6 — a), 


For this force field the work depends only on the end points a and 6 and not on the curve 
joining them. Not all force fields have this property. Those which do are called conservative. 
The example on p. 325 is a nonconservative force field. In a later section we shall determine 
all conservative force fields. 


EXAMPLE 2. The principle of work and energy. A particle of mass m moves along a curve 
under the action of a force field f. If the speed of the particle at time f¢ is v(t), its kinetic 
energy is defined to be $mv*(t). Prove that the change in kinetic energy in any time interval 
is equal to the work done by f during this time interval. 


Solution. Let r(t) denote the position of the particle at time ¢. The work done by f 
during a time interval [a, b] is {712} f- dr. We wish to prove that 


ae P 5 ; r 
ay dr = 4mv‘(b) — 4mv‘*(a). 


r 


From Newton’s second law of motion we have 
f[r(t)] = mr'(t) = mv'(t), 


where v(t) denotes the velocity vector at time ft. The speed is the length of the velocity 
vector, v(t) = ||v(t)||. Therefore 


flr) °°) = Flr@] - 0) = mv'(t) - v(t) = gm = (v(t) - v(t)) = gm “ (0%(9). 
Integrating from a to b we obtain 


of de = f° fle): '@ dt = gme%(o], = pmo") — dmv%(a), 


as required. 


10.7 Line integrals with respect to arc length 


Let @ be a path with @ continuous on an interval [a, b]. The graph of @ is a rectifiable 
curve. In Volume I we proved that the corresponding arc-length function s is given by the 
integral 


t 
s(t) = | lla'(u)ll du. 
The derivative of arc length is given by 


s(t) = |e’ (oll. 
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Let y be a scalar field defined and bounded on C, the graph of a. The ine integral of » 
with respect to arc length along C is denoted by the symbol J¢ @ ds and is defined by the 
equation 


[eds = ]° plans’ dt, 


whenever the integral on the right exists. 

Now consider a scalar field » given by p[a(t)] = f[a(t)]- T(t), the dot product of a 
vector field f defined on C and the unit tangent vector T(t) = (da/ds). In this case the line 
integral {¢ ds is the same as the line integral J. f- da because 


da ds 


fla(t)]: @'(t) = fla(s)) - poe = fla(t)] - T)s'(t) = gla()]s‘(). 


When f denotes a velocity field, the dot product f- T is the tangential component of 
velocity, and the line integral J. f- T ds is called the flow integral of falong C. When Cisa 
closed curve the flow integral is called the circulation of falong C. These terms are commonly 
used in the theory of fluid flow. 


10.8 Further applications of line integrals 


Line integrals with respect to arc length also occur in problems concerned with mass 
distribution along a curve. For example, think of a curve C in 3-space as a wire made of a 
thin material of varying density. Assume the density is described by a scalar field », where 
p(x, y, Z) is the mass per unit length at the point (x, y, z) of C. Then the total mass M of 
the wire is defined to be the line integral of » with respect to arc length: 


M= [. G(x, y, Z) ds. 


The center of mass of the wire is defined to be the point (x, ~, Z) whose coordinates are 
determined by the equations 


xM = iP x(x, y, z) ds, yM = I. y@(x, y, z) ds, ZM = is z9(x, y, z) ds. 


A wire of constant density is called uniform. In this case the center of mass is also called the 
centroid. 


EXAMPLE 1. Compute the mass M of one coil of a spring having the shape of the helix 
whose vector equation is 
a(t)=acosti+asintj + btk 
if the density at (x, y, z) is x? + y® 4+ z?. 


Solution. The integral for M is 


M = i (P+ y + 2%) ds= i (a® cos? t + a? sin? t + b°t”)s’(t) dt. 
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Since s‘(t) = ||a’(t)|| and a(t) = —asinti+ acostj + bk, we have s’(t) = Ja + b? 
and hence 


pe, 2r a eee 
Maes: | (a? +B) dt = Va? + B (27a! < 5 7b") | 
0 


In this example the z-coordinate Z of the center of mass is given by 


2M = i. 2(x* + y? + 2°) ds = Va? + BP i bt(a® + b?t*) dt 
= ba? + b® (27a? + 4r'b?). 
The coordinates x and jy are requested in Exercise 15 of Section 10.9. 


Line integrals can be used to define the moment of inertia of a wire with respect to an 
axis. If d(x, y, z) represents the perpendicular distance from a point (x, y, z) of C to an 
axis L, the moment of inertia /;, is defined to be the line integral 


Ir, = i é*(x, y> Z) Q(x, y> Z) ds , 


where (x, y, Zz) is the density at (x, y, z). The moments of inertia about the coordinate 
axes are denoted by /,, /,, and J,. 


EXAMPLE 2. Compute the moment of inertia J, of the spring coil in Example 1. 


Solution. Here 0?(x, y,z) =x? + y? =a’ and ¢(x, y, z) = x? + y? + z?, so we have 


I,= [. (x? + y*\(x? + y*? + z*) ds =a’ {, (x? + y? + z*) ds = Ma’, 


where M is the mass, as computed in Example 1. 


10.9 Exercises 


1. A force field f in 3-space is given by f(x, y, z) = xi + yj + (xz — y)k. Compute the work 
done by this force in moving a particle from (0, 0, 0) to (1, 2, 4) along the line segment joining 
these two points. 

2. Find the amount of work done by the force f(x, y) = (x — y”)i + 2xyj in moving a particle 
(in a counterclockwise direction) once around the square bounded by the coordinate axes and 
the lines x =aandy =a,a>0. 

3. A two-dimensional force field f is given by the equation f(x, y) = cxyi + x®y®j, where c is a 
positive constant. This force acts on a particle which must move from (0, 0) to the line x = 1 
along a curve of the form 


y=ax’, where a>O and 6>0. 


Find a value of a (in terms of c) such that the work done by this force is independent of b. 

4. A force field fin 3-space is given by the formula f(x, y, z) = yzi + xzj + x(y + Ik. Calculate 
the work done by fin moving a particle once around the triangle with vertices (0, 0, 0), (1, 1, 1), 
(—1,1, —1) in that order. 
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. Calculate the work done by the force field f(x, y,z) = (y —zi+ (z —x¥i +(x - y)k along 


the curve of intersection of the sphere x? + y* + z? = 4 and the plane z = y tan 0, where 
0 <6 < 7/2. The path is transversed in a direction that appears counterclockwise when 
viewed from high above the xy-plane. 


. Calculate the work done by the force field f(x, y, z) = yitzy + xk along the curve of 


intersection of the sphere x? + y? + z? = a? and the cylinder x? + y? = ax, where z > 0 and 
a>0O. The path is traversed in a direction that appears clockwise when viewed from high 
above the xy-plane. 


Calculate the line integral with respect to arc length in each of Exercises 7 through 10. 


ll. 


16. 


. Jo (« + y) ds, where C is the triangle with vertices (0, 0), (1, 0), and (0, 1), traversed in a 


counterclockwise direction. 


. Jc y® ds, where C has the vector equation 


a(t) = a(t —sint)i + a(l1 —cost)j, 0O<t <27. 


. Jo (@? + y*) ds, where C has the vector equation 


a(t) =a(cost + ¢tsint)i + a(sint — tcost)j, 0<t<2rz. 


. Jo z ds, where C has the vector equation 


a(t) =fcosti+tsintj + tk, O<t<t. 


Consider a uniform semicircular wire of radius a. 

(a) Show that the centroid lies on the axis of symmetry at a distance 2a/7 from the center. 
(b) Show that the moment of inertia about the diameter through the end points of the wire 
is }Ma?, where M is the mass of the wire. 


. A wire has the shape of the circle x? + y? = a*. Determine its mass and moment of inertia 


about a diameter if the density at (x, y) is |x] + lyl. 


. Find the mass of a wire whose shape is that of the curve of intersection of the sphere x? + y? + 


z* = | and the plane x + y + z = Oif the density of the wire at (x, y, z) is x”. 


. A uniform wire has the shape of that portion of the curve of intersection of the two surfaces 


x* + y® = z*and y? = x connecting the points (0, 0, 0) and (1, 1, . 2). Find the z-coordinate 
of its centroid. 


. Determine the coordinates x and ¥ of the center of mass of the spring coil described in Example 


1 of Section 10.8. 
For the spring coil described in Example 1 of Section 10.8, compute the moments of inertia J, 


and I,. 


10.10 Open connected sets. Independence of the path 


Let S be an open set in R”. The set S is called connected if every pair of points in S can be 
joined by a piecewise smooth path whose graph lies in S. That is, for every pair of points a 
and b in S there is a piecewise smooth path a@ defined on an interval [a, b] such that a(t)e S 


for 


each tin [a, b], with a(a) = a and a(b) = b. 


Three examples of open connected sets in the plane are shown in Figure 10.3. Examples 
in 3-space analogous to these would be (a) a solid ellipsoid, (b) a solid polyhedron, and (c) 
a solid torus; in each case only the interior points are considered. 
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An open set S is said to be disconnected if S is the union of two or more disjoint non- 
empty open sets. An example is shown in Figure 10.4. It can be shown that the class of 
open connected sets is identical with the class of open sets that are not disconnected. 

Now let f be a vector field that is continuous on an open connected set S. Choose two 
points a and b in S and consider the line integral of f from a to b along some piecewise 
smooth path in S. The value of the integral depends, in general, on the path joining a to b. 
For some vector fields, the integral depends only on the end points a and 6 and not on the 
path which joins them. In this case we say the integral is independent of the path froma to b. 
We say the line integral of fis independent of the path in Sif it is independent of the path from 
a to b for every pair of points a and bin S. 


(a) (b) : 
Ficure 10.3 Examples of open connected sets. FiGurE 10.4 A disconnected set S, the 
union of two disjoint circular disks. 


Which vector fields have line integrals independent of the path? To answer this question 
we extend the first and second fundamental theorems of calculus to line integrals. 


10.11 The second fundamental theorem of calculus for line integrals 


The second fundamental theorem for real functions, as proved in Volume I (Theorem 
5.3), states that 


[? oO dt = ob) — Ca), 


provided that g’ is continuous on some open interval containing both a and b. To extend 
this result to line integrals we need a slightly stronger version of the theorem in which 
continuity of m’ is assumed only in the open interval (a, b). 


THEOREM 10.2. Let be a real function that is continuous on a closed interval {a, b] and 
assume that the integral |° y'(t) dt exists. If y’ is continuous on the open interval (a, b), we 
have 


J o'W dt = 9b) — 9(a). 


Proof. For each x in [a, b] define f(x) = J* y(t) dt. We wish to prove that 


(10.2) (6) = v(b) — (a). 


+ For a further discussion of connectedness, see Chapter 8 of the author’s Mathematical Analysis, Addison- 
Wesley, Reading, Mass., 1957. 
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By Theorem 3.4 of Volume I, fis continuous on the closed interval [a, b]. By Theorem 5.1 
of Volume I, fis differentiable on the open interval (a, b), with f’(x) = @‘(x) for each x 
in (a,b). Therefore, by the zero-derivative theorem (Theorem 5.2 of Volume I), the 
difference f — @ is constant on the open interval (a,b). By continuity, f— 9 is also 
constant on the closed interval [a,b]. In particular, f(b) — 9(b) = f(a) — (a). But 
since f(a) = 0, this proves (10.2). 


THEOREM 10.3. SECOND FUNDAMENTAL THEOREM OF CALCULUS FOR LINE INTEGRALS. 
Let @ be a differentiable scalar field with a continuous gradient Vg on an open connected set 
Sin R". Then for any two points a and b joined by a piecewise smooth path a in S we have 


[- vq + da = (6) — oa). 


Proof. Choose any two points a and bin S and join them by a piecewise smooth path a 
in S defined on an interval [a,b]. Assume first that a is smooth on [a,b]. Then the line 
integral of Vm from a to b along a is given by 


b b 
| Vo:da= : Vgy[a(t)] > a’(t) dt. 


By the chain rule we have 
Vela(t)] a(t) =g°(), 


where g is the composite function defined on [a, b] by the formula 


g(t) = pla(t)]. 


The derivative g’ is continuous on the open interval (a, b) because V@ is continuous on S 
and «is smooth. Therefore we can apply Theorem 10.2 to g to obtain 


[, Vo “da = [’ g'(t) dt = g(b) — g(a) = yla(b)] — yla(a)] = 9(b) — ea). 


This proves the theorem if @ is smooth. 

When @ is piecewise smooth we partition the interval [a, b] into a finite number (say r) 
of subintervals [t,_,, ¢,], in each of which @ is smooth, and we apply the result just proved 
to each subinterval. This gives us 


[vo = [OP Vo =S {olace)] — olatt, I} = 918) — 90), 


a(ty—1) 
as required. 


As a consequence of Theorem 10.3 we see that the line integral of a gradient is inde- 
pendent of the path in any open connected set S in which the gradient is continuous. Fora 
closed path we have b = a, so y(b) — y(a) = 0. In other words, the line integral of a 
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continuous gradient is zero around every piecewise smooth closed path in S. \n Section 10.14 
we shall prove (in Theorem 10.4) that gradients are the on/y continuous vector fields with 
this property. 


10.12 Applications to mechanics 


If a vector field fis the gradient of a scalar field y, then ¢ is called a potential function 
for f. In 3-space, the level sets of m are called equipotential surfaces; in 2-space they are 
called equipotential lines. (If g denotes temperature, the word “‘equipotential”’ is replaced 
by “‘isothermal’’; if g denotes pressure the word “‘isobaric’’ is used.) 


EXAMPLE 1. In 3-space, let g(x, y, z) =r", where r = (x? + y? + z?)%. For every 
integer n we have 
V(r") = ar" r, 


where r = xi + yj + zk. (See Exercise 8 of Section 8.14.) Therefore is a potential of 
the vector field 


S(x,y, Z) = nr". 
The equipotential surfaces of g are concentric spheres centered at the origin. 


EXAMPLE 2. The Newtonian potential. Newton’s gravitation law states that the force f 
which a particle of mass M exerts on another particle of mass m is a vector of length 
GmM/r*, where G is a constant and r is the distance between the two particles. Place the 
origin at the particle of mass M, and let r = xi + yj + zk be the position vector from the 
origin to the particle of mass m. Then r = |r|| and —r/r is a unit vector with the same 
direction as f, so Newton’s law becomes 


f= —GmMr-*r. 


Taking n = —1 in Example 1 we see that the gravitational force fis the gradient of the 
scalar field given by 


Q(x, y, Z) = GmMr™. 


This is called the Newtonian potential. 
The work done by the gravitational force in moving the particle of mass m from 
(X1, Vi» Z1) to (X2, Ve, Ze) Is 


1 1 
Q(x, Yi 2) — P(Xe, Yo5Z) = Gmo (~ as | ) 


ry i) 


where r, = (x? + y? + 27)“ and r, = (x2 + y3 + 22)’*. If the two points lie on the same 
equipotential surface then r; = r, and no work is done. 


EXAMPLE 3. The principle of conservation of mechanical energy. Let f be acontinuous force 
field having a potential m in an open connected set S. Theorem 10.3 tells us that the work 
done by f in moving a particle from a to x along any piecewise smooth path in S is 
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p(x) — (a), the change in the potential function. In Example 2 of Section 10.6 we proved 
that this work is also equal to the change in kinetic energy of the particle, k(x) — k(a) 
where k(x) denotes the kinetic energy of the particle when it is located at x. Thus, we have 


k(x) — k(a) = 9(x) — ¢(a), 
Or 


(10.3) k(x) — (x) = k(a) — 9@). 


The scalar — g(x) is called the potential energy} of the particle. 

If ais kept fixed and x is allowed to vary over the set S, Equation (10.3) tells us that the 
sum of k(x) and — q(x) is constant. In other words, if a force field is a gradient, the sum 
of the kinetic and potential energies of a particle moving in this field is constant. In mechanics 
this is called the principle of conservation of (mechanical) energy. A force field with a 
potential function is said to be conservative because the total energy, kinetic plus potential, 
is conserved. In a conservative field, no net work is done in moving a particle around a 
closed curve back to its starting point. A force field will not be conservative if friction or 
viscosity exists in the system, since these tend to convert mechanical energy into heat 
energy. 


10.13 Exercises 


1. Determine which of the following open sets S in R? are connected. For each connected set, 
choose two arbitrary distinct points in S and explain how you would find a piecewise smooth 
curve in S connecting the two points. 

(a) S = {(x, y)| x7 + y? > 0}. (c) S={(x,y)| x7 + y? < 1}. 
(b) S={(x,y)|x?>+y? >0}. (d) S={(x, y)|1 <x? + y? <2}. 
(ec) S={(x,y)|x+y? >1 and (x —3)? +y? > 1}. 
(f) S={(x,y)|x?+y? <1 or (x —3)? +7 < ]}. 
2. Given a two-dimensional vector field 


f(x, y) = P(x, y)i + O(x, yi, 


where the partial derivatives 0P/dy and 0Q/@x are continuous on an open set S. If fis the 
gradient of some potential y, prove that 


aP aQ 
ay Ox 
at each point of S. 
3. For each of the following vector fields, use the result of Exercise 2 to prove that f is not a 
gradient. Then find a closed path C such that ¢, f #0. 
(a) f(x,y) = yi — xf. 
(b) f(x, y) = yi + (xy — x). 


4. Given a three-dimensiona!l vector field 


f(x, y, Z) = P(x, y, z)i + O(x, y, zi + R(x, y, zk, 


+ Some authors refer to —¢ as the potential function of fso that the potential energy at x will be equal to the 
value of the potential function » at x. 


10. 
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where the partial derivatives 
oP OP 0Q 0Q @R OR 
ay’ as aye? ao? ax’ ay’ 


are continuous on an open set S. If fis the gradient of some potential function ¢, prove that 


aP  aQ aP aR 20 «AR 


dy ax’ dz ax’ az ay 
at each point of S. 


. For each of the following vector fields, use the result of Exercise 4 to prove that fis not a 


gradient. Then find a closed path C such that $, f #0. 
(a) f(x, y, Zz) = yi + xf + xk. 
(b) f(x, y, z) = xyi + (x? + Lj + 27k. 


. A force field fis defined in 3-space by the equation 


f(x, y,z) =yi + at yzk. 


(a) Determine whether or not f is conservative. 
(b) Calculate the work done in moving a particle along the curve described by 


a(t) =costi+sin¢tj + ek 
as truns from 0 to =z. 


. A two-dimensional force field F is described by the equation 


F(x, y) = (x + ylit+ (x — yp). 
(a) Show that the work done by this force in moving a particle along a curve 
a(t) =f(hitg@, a<t<ob, 


depends only on f(a), f(5), g(a), g (). 
(b) Find the amount of work done when f(a) = 1, f(b) = 2, g(a) = 3, g(b) = 4. 


. A force field is given in polar coordinates by the equation 


F(r,6) = —4 sin 0% + 4 sin 67. 


Compute the work done in moving a particle from the point (1, 0) to the origin along the spiral 
whose polar equation isr = e®. 


. A radial or “‘central’’ force field F in the plane can be written in the form F(x, y) = f(r)r, 


where r = xi + yjandr = |/r||. Show that such a force field is conservative. 

Find the work done by force F(x, y) = (3y? + 2)i + 16xj. in moving a particle from (—1, 0) 
to (1, 0) along the upper half of the ellipse b?x? + y? = b?. Which ellipse (that is, which 
value of 5) makes the work a minimum? 


10.14 The first fundamental theorem of calculus for line integrals 


Section 10.11 extended the second fundamental theorem of calculus to line integrals. 


This section extends the first fundamental theorem. We recall that the first fundamental 
theorem states that every indefinite integral of a continuous function f has a derivative 
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equal to f. That is, if 
wo) = JF fat, 


then at the points of continuity of f we have 


y (x) = f(x). 


To extend this theorem to line integrals we begin with a vector field f, continuous on an 
open connected set S, and integrate it along a piecewise smooth curve C from a fixed 
point ain Sto an arbitrary point x. Then we let ~ denote the scalar field defined by the line 
integral 


y(x) = | f- dee, 


where @ describes C. Since S is connected, each point x in S can be reached by such a 
curve. For this definition of g(x) to be unambiguous, we need to know that the integral 
depends only on x and not on the particular path used to joinato x. Therefore, it is natural 
to require the line integral of fto be independent of the path in S. Under these conditions, 
the extension of the first fundamental theorem takes the following form: 


THEOREM 10.4. FIRST FUNDAMENTAL THEOREM FOR LINE INTEGRALS. Let f be a vector 
field that is continuous on an open connected set S in R", and assume that the line integral of f 
is independent of the path in S. Let a be a fixed point of S and define a scalar field p on S by 
the equation 


wx) = fpf de, 


where @ is any piecewise smooth path in S joining a to x. Then the gradient of 9 exists and is 
equal to f; that is, 


Vo(x) =f(x)  forevery xin S. 


Proof. We shall prove that the partial derivative D,@(x) exists and is equal to f,(x), 
the Ath component of f(x), foreach k = 1, 2,..., mand each xin S. 

Let B(x; r) be an n-ball with center at x and radius r lying in S. If y is a unit vector, the 
point x + hy also lies in S for every real A satisfying 0 < |h| <r, and we can form the 
difference quotient 


p(x + hy) — 9(x) 
a 


Because of the additive property of line integrals, the numerator of this quotient can be 
written as 


p(x + hy) — g(x) = ["'F- da, 


and the path joining x to x + Ay can be any piecewise smooth path lying in S. In particular, 
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we can use the line segment described by 
a(t) = x + thy, where 0<t<l. 


Since a’(t) = hy, the difference quotient becomes 


Q(x + 2 — 9(x) = [ s+ thy): y dt. 


(10.4) 
Now we take y = e,, the kth unit coordinate vector, and note that the integrand becomes 
f(x + thy): y =f, (x + the,). Then we make the change of variable u = ht, du =h dt, 
and we write (10.4) in the form 


(10.5) eens = ~ [ te + ue,) du = mae 


where g is the function defined on the open interval (—r, r) by the equation 


g(t) = |p fale + ney) du. 


Since each component /, is continuous on S, the first fundamental theorem for ordinary 
integrals tells us that g’(t) exists for each ¢ in (—r, r) and that 


g(t) =Si(x + te,). 


In particular, ¢'(0) = f,(x). Therefore, if we let h — 0 in (10.5) we find that 


lim G(x + he,) — 9) = lim g(h) — (0) = g'(0) = f,(x). 


This proves that the partial derivative D, g(x) exists and equals f,(x), as asserted. 


10.15 Necessary and sufficient conditions for a vector field to be a gradient 


The first and second fundamental theorems for line integrals together tell us that a 
necessary and sufficient condition for a continuous vector field to be a gradient on an open 
connected set is for its line integral between any two points to be independent of the path. 
We shall prove now that this condition is equivalent to the statement that the line integral 
is zero around every piecewise smooth closed path. All these conditions are summarized in 
the following theorem. 


THEOREM 10.5. NECESSARY AND SUFFICIENT CONDITIONS FOR A VECTOR FIELD TO BE A 
GRADIENT. Let f be a vector field continuous on an open connected set S in R". Then the 
following three statements are equivalent. 

(a) fis the gradient of some potential function in S. 

(b) The line integral of f is independent of the path in S. 

(c) The line integral of f is zero around every piecewise smooth closed path in S. 
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Proof. We shall prove that (b) implies (a), (a) implies (c), and (c) implies (b). State- 
ment (b) implies (a) because of the first fundamental theorem. The second fundamental 
theorem shows that (a) implies (c). 

To complete the proof we show that (c) implies (b). Assume (c) holds and let C, and C, 
be any two piecewise smooth curves in S with the same end points. Let C, be the graph ofa 
function a defined on an interval [a, b], and let C, be the graph of a function B defined on 
[c, d]. 


Define a new function y as follows: 


a(t) if a<r<b, 


10={o ae if b<t<b+d-e. 


Then y describes a closed curve C such that 
i? = | fda Lf ae 


Since ¢,f * dy = 0 because of (c), we have of: da = feSt° dB, so the integral of f is 
independent of the path. This proves (b). Thus, (a), (b), and (c) are equivalent. 


Note: If bo f #O0fora particular closed curve C, then fis not a gradient. However, 
if a line integral $, f is zero for a particular closed curve C or even for infinitely many 
closed curves, it does not necessarily follow that fis a gradient. For example, the reader 
can easily verify that the line integral of the vector field f(x, y) = xi + xyj is zero for every 
circle C with center at the origin. Nevertheless, this particular vector field is not a 
gradient. 


10.16 Necessary conditions for a vector field to be a gradient 


The first fundamental theorem can be used to determine whether or not a given vector 
field is a gradient on an open connected set S. If the line integral of fis independent of the 
path in S, we simply define a scalar field g by integrating f from some fixed point to an 
arbitrary point x in S along a convenient path in S. Then we compute the partial derivatives 
of g and compare D,¢ with f,, the kth component of f. If D, v(x) = f,(x) for every x in 
S and every k, then fis a gradient on S and ¢ isa potential. If D,g(x) ¥ f,(x) for some k 
and some x, then fis not a gradient on S. 

The next theorem gives another test for determining when a vector field fis not a gradient. 
This test is especially useful in practice because it does not require any integration. 


THEOREM 10.6. NECESSARY CONDITIONS FOR A VECTOR FIELD TO BE A GRADIENT. Let 
f=(fh,...,f,) be a continuously differentiable vector field on an open set SinR”. If fis a 
gradient on S, then the partial derivatives of the components of f are related by the equations 


(10.6) Di f(x) = Dj fix) 


fori,j=1,2,...,nand every x in S. 
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Proof. If fis a gradient, then f= Vq@ for some potential function g. This means that 


fi, = Di 
foreach j = 1,2,...,m. Differentiating both members of this equation with respect to x; 
we find 
Df; = D:D;¢. 
Similarly, we have 
Df; eal D,D:¢ . 


Since the partial derivatives D;f; and D,f; are continuous on S, the two mixed partial 
derivatives D;D,y and D,;D;p must be equal on S. This proves (10.6). 


EXAMPLE |. Determine whether or not the vector field 
F(x, y) = 3x*yt + x*yj 
is a gradient on any open subset of R?. 
Solution. Here we have 
filx, y) = 3x?y, f(x, y) = x*y. 
The partial derivatives D,f, and D, fy are given by 
Dofix,y) = 3x, Dyfolx, y) = 3x°y. 


Since Do fi(x, y) ¥ Di fo(x, y) except when x = 0 or y= 1, this vector field is not a 
gradient on any open subset of R?. 


The next example shows that the conditions of Theorem 10.6 are not always sufficient 
for a vector field to be a gradient. 


EXAMPLE 2. Let S be the set of all (x, y) ¥ (0, 0) in R?, and let f be the vector field 
defined on S by the equation 


— rg x , 
Xs = i+ ‘ 


Show that D,f, = D,f, everywhere on S but that, nevertheless, fis not a gradient on S. 


Solution. The reader can easily verify that D,f.(x, y) = D.f\(x, y) for all (x, y) in S. 
(See Exercise 17 in Section 10.18.) 

To prove that fis not a gradient on S we compute the line integral of f around the unit 
circle given by 


a(t) =cosri+ sintj, 0<t<27z. 
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We obtain 
$f: da =|" fla(t)] a'(0) dt = |" (sin®t + cos® t) dt = 2. 


Since the line integral around this closed path is not zero, fis not a gradient on S. Further 
properties of this vector field are discussed in Exercise 18 of Section 10.18. 


At the end of this chapter we shall prove that the necessary conditions of Theorem 10.6 
are also sufficient if they are satisfied on an open convex set. (See Theorem 10.9.) 


10.17 Special methods for constructing potential functions 


The first fundamental theorem for line integrals also gives us a method for constructing 
potential functions. If fis a continuous gradient on an open connected set S, the line 
integral of fis independent of the path in S. Therefore we can find a potential » simply by 
integrating f from some fixed point a to an arbitrary point x in S, using any piecewise 
smooth path lying in S. The scalar field so obtained depends on the choice of the initial 
point a. If we start from another initial point, say b, we obtain a new potential y. But, 
because of the additive property of line integrals, g and py can differ only by a constant, 
this constant being the integral of ffrom a to b. 

The following examples illustrate the use of different choices of the path of integration. 


FiGurE 10.5 Two polygonal paths from (a, 5) to (x, y). 


EXAMPLE |. Construction of a potential on an open rectangle. If fis a continuous gradient 
on an open rectangle in R”, a potential » can be constructed by integrating from a fixed 
point to an arbitrary point along a set of line segments parallel to the coordinate axes. 
A two-dimensional example is shown in Figure 10.5. We can integrate first from (a, b) to 
(x, b) along a horizontal segment, then from (x, 5) to (x, y) along a vertical segment. 
Along the horizontal segment we use the parametric representation 


a(t) = ti+ bj, ax<t<x, 
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and along the vertical segment we use the representation 
a(t)=xi+ 4, b<t<y. 


If f(x, y) =fA(x, yi + A(x, y)j, the resulting formula for a potential p(x, y) is 


(10.7) oO) = [Ale bat + JP AG ode, 


We could also integrate first from (a, 5) to (a, y) along a vertical segment and then from 
(a, y) to (x, y) along a horizontal segment as indicated by the dotted lines in Figure 10.5. 
This gives us another formula for (x, y), 


(10.8) ox,» = | Ala nat + J °F, y)dt. 


Both formulas (10.7) and (10.8) give the same value for y(x, y) because the line integral 
of a gradient is independent of the path. 


EXAMPLE 2. Construction of a potential function by the use of indefinite integrals. The use 
of indefinite integrals often helps to simplify the calculation of potential functions. For 
example, suppose a three-dimensional vector field f= (/,,f2,/5) is the gradient of a 
potential function @ on an open set S in R*. Then we have 


Op _ Op _ Op _ 
ay I ay Pe az Is 


everywhere on S. Using indefinite integrals and integrating the first of these equations with 
respect to x (holding y and z constant) we find 


(x, y, Z) = | fal, y, z) dx + Aly, z), 


where A(y, z) is a “constant of integration” to be determined. Similarly, if we integrate the 
equation 0g/dy = f, with respect to y and the equation 0y/dz = f, with respect to z we 
obtain the further relations 


p(x, yz) = | fal, y, 2) dy + BCs, 2) 
and 


ox, ys 2) = | fulx ys 2) dz + Clx, y), 


where B(x, z) and C(x, y) are functions to be determined. Finding ¢ means finding three 
functions A(y, z), B(x, z), and C(x, y) such that all three equations for (x, y, z) agree in 
their right-hand members. In many cases this can be done by inspection, as illustrated by 
the following example. 
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EXAMPLE 3. Find a potential function @ for the vector field defined on R® by the equation 
S(x,y, Z) = (Qxyz + 2% — 2y? + Di + (x72 — 4xy)j + (x?y + 2xz — 2)k. 


Solution. Without knowing in advance whether or not fhas a potential function ~, we 
try to construct a potential as outlined in Example 2, assuming that a potential @ exists. 
Integrating the component /; with respect to x we find 


Q(x, y, Z) = | xyz + 27 — 2y? + 1) dx + A(y, z) = x?yz + xz” — 2xy® + x + A(y,z). 


Integrating f, with respect to y, and then /; with respect to z, we find 


g(x, y, Z) = [ G2 — 4xy) dy + B(x, z) = x*yz — 2xy* + B(x, z), 
Q(x, y, Z) = | Gy + 2xz — 2) dz + C(x, y) = x*yz + xz” — 2z + C(x, y). 


By inspection we see that the choices A(y, z) = —2z, B(x, z) = xz? +x —2z, and 
C(x, y) = x — 2xy? will make all three equations agree; hence the function @ given by the 
equation 

Q(X, y, Z) = x*yz + xz* — Qxy? ++ x — 22 


is a potential for fon R?. 


EXAMPLE 4. Construction of a potential on a convex set. A set S in R” is called convex 
if every pair of points in S can be joined by a line segment, all of whose points lie in S. 
An example is shown in Figure 10.6. Every open convex set is connected. 

If fis a continuous gradient on an open convex set, then a potential g can be constructed 
by integrating f from a fixed point a in S to an arbitrary point x along the line segment 
joining a to x. The line segment can be parametrized by the function 


a(t)=a-+ t(x —a), Where 0<t< 1. 


This gives us a(t) = x — a, so the corresponding potential is given by the integral 


(10.9) p(x) = | fla + (x — a) (x — a) dt. 


Convex Not convex 


FIGURE 10.6 In aconvex set S, the segment joining a and bis in S for all points a and 
bin S. 
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If S contains the origin we can take a = O and write (10.9) more simply as 
1 
(10.10) p(x) = | flix) x dt. 


10.18 Exercises 


In each of Exercises 1 through 12, a vector field fis defined by the formulas given. In each case 
determine whether or not f is the gradient of a scalar field. When fis a gradient, find a correspond- 
ing potential function 9. 


1. f(x,y) =xit+ yj. 

2. f(x, y) = 3x*yi + xf. 

3. f(x, y) = (2xe” + y)i + (x7eY + x — 2y)j. 

4. f(x, y) = (siny — ysinx + x)i + (cosx +xcosy + y)j. 

5. f(x, y) = [sin (xy) + xy cos (xy)]i + x? cos (xy)j. 

6. f(x, y, Zz) = xi + yj + Zk. 

7. f(x, y,z7) =(xX + z)i-— (y + Zh + (x — yk. 

8. f(x, y, Z) = 2xy*'l + x?z3j + 3x*yz*k. 

9. f(x, y, Zz) = 3y1z7i + 4x327j — 3x*y?k. 

10. f(x, y, z) = (2x7 + 8xy")i + (3x3y — 3xy)j — (4y2z? + 2x°z)k. 
11. f(x, y,z) = (cos x + 23) — (4 — 2y sin x)j + (3xz? + 2)k. 
12. f(x, y, Zz) = (4xy — 3x2z2 + Li + 2(x? + Dj — (2x°2 + 32*)k. 


13. A fluid flows in the xy-plane, each particle moving directly away from the origin. Ifa particle 
is at a distance r from the origin its speed is ar”, where a and n are constants. 
(a) Determine those values of a and n for which the velocity vector field is the gradient of 
some scalar field. 
(b) Find a potential function of the velocity whenever the velocity is a gradient. The case 
n = —1 should be treated separately. 

14. If both and y are potential functions for a continuous vector field f on an open connected 
set S in R”, prove that » — y is constant on S. 

15. Let S be the set of allx # Oin R”. Let r = ||x||, and let f be the vector field defined on S by 
the equation 


f(x) =r?x, 


where p is a real constant. Find a potential function for fon S. The case p = —2 should be 
treated separately. 
16. Solve Exercise 15 for the vector field defined by the equation 


©) 
f(x) <2 x, 
r 
where g is a real function with a continuous derivative everywhere on R’. 
The following exercises deal with the vector field f defined on the set S of all points (x, y) ¥ (0, 0) 


in R? by the equation 


— ne A pee ae 
f(x, y) = x2 +y x2 + yl" 
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In Example 2 of Section 10.16 we showed that f is not a gradient on S, even though D,f, = D2; 
everywhere on S. 


17. Verify that for every point (x, y) in S we have 


2 2 


yuax 


D,fo(x, y) = Defi, y) = G2 + ye 


18. This exercise shows that fis a gradient on the set 
T = R* - {(x,y)|y =0, x <0}, 


consisting of all points-in the xy-plane except those on the nonpositive x-axis. 
(a) If (x, y) € T, express x and y in polar coordinates, 


x =rcos 08, y=rsin6, 


where r > Oand —7 < 6 <7. Prove that 6 is given by the formulas 


arctan = if x >0, 


1 
2 if x =0, 


arctan +7 if x <0. 


[Recall the definition of the arc tangent function: For any real ¢, arctan ¢ is the unique real 
number ¢ which satisfies the two conditions tan y = tand —72/2 < 9» < 2/2.] 


(b) Deduce that 
00 y 00 x 


Ray) Ty 


for all (x, y) in T. This proves that @ is a potential function for fon the set T. 
This exercise illustrates that the existence of a potential function depends not only on the 
vector field f but also on the set in which the vector field is defined. 


10.19 Applications to exact differential equations of first order 


Some differential equations of the first order can be solved with the aid of potential 
functions. Suppose we have a first-order differential equation of the form 


y = f(x, y). 


If we multiply both sides by a nonvanishing factor Q(x, y) we transform this equation 
to the form Q(x, y)y’ — f(x, y)QO(x, y) = 0. If we then write P(x, y) for —f(x, y)Q(x, y) 
and use the Leibniz notation for derivatives, writing dy/dx for y’, the differential equation 
takes the form 


(10.11) P(x, y) dx + Q(x, y) dy = 0. 
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We assume that P and Q are continuous on some open connected set S in the plane. 
With each such differential equation we can associate a vector field V, where 


V(x, y) = P(x, ylit Q(x, y)j. 


The components P and Q are the coefficients of dx and dy in Equation (10.11). The differ- 
ential equation in (10.11) is said to be exact in S if the vector field V is the gradient of a 
potential; that is, if V(x, y) = V(x, y) for each point (x, y) in S, where ¢ is some scalar 
field. When such a exists we have 0qg/0x = P and 0g/dy = Q, and the differential 
equation in (10.11) becomes 


og dx + hi dy =0. 

Ox Oy 

We shall prove now that each solution of this differential equation satisfies the relation 
g(x, y) = C, where C is a constant. More precisely, assume there is a solution Y of the 
differential equation (10.11) defined on an open interval (a, b) such that the point (x, Y(x)) 
is in S for each x in (a, b). We shall prove that 


glx, Y(x)J=C 


for some constant C. For this purpose we introduce the composite function g defined on 
(a, b) by the equation 


g(x) = lx, ¥Q)]. 
By the chain rule, the derivative of g is given by 
(10.12) g(x) = Digp[x, Y@)] + Daplx, Y()Y"(x) = P(x, y) + O(x, yyy’, 
where y = Y(x) and y' = Y"(x). If y satisfies (10.11), then P(x, y) + Q(x, y)y’ = 0, so 
g (x) = 0 for each x in (a, b) and, therefore, g is constant on (a, b). This proves that every 
solution y satisfies the equation g(x, y) = C. 


Now we may turn this argument around to find a solution of the differential equation. 
Suppose the equation 


(10.13) p(x, y) = C 
defines y as a differentiable function of x, say y = Y(x) for x in an interval (a, b), and 
let g(x) = y[x, Y(x)]. Equation (10.13) implies that g is constant on (a, 6). Hence, by 


(10.12), P(x, y) + O(x, y)y’ =0, so y is a solution. Therefore, we have proved the 
following theorem: 


THEOREM 10.7. Assume that the differential equation 


(10.14) P(x, y) dx + Q(x, y) dy =0 
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is exact in an open connected set S, and let » be a scalar field satisfying 


ey = and he = Q 
Ox Oy 


everywhere in S. Then every solution y = Y(x) of (10.14) whose graph lies in S satisfies the 
equation y[x, Y(x)] = C for some constant C. Conversely, if the equation 


P(x, y)=C 
defines y implicitly as a differentiable function of x, then this function is a solution of the 


differential equation (10.14). 


The foregoing theorem provides a straightforward method for solving exact differential 
equations of the first order. We simply construct a potential function » and then write 
the equation y(x, y) = C, where C is a constant. Whenever this equation defines y 
implicitly as a function of x, the corresponding y satisfies (10.14). Therefore we can use 
Equation (10.13) as a representation of a one-parameter family of integral curves. Of 
course, the only admissible values of C are those for which (x9, yo) = C for some (Xo, Yo) 
in S. 


EXAMPLE |. Consider the differential equation 


dy __ 3x" + 6xy’ 
dx 6x*y + 4y3- 


Clearing the fractions we may write the equation as 
(3x? + 6xy?) dx + (6x?y + 4y*) dy = 0. 
This is now a special case of (10.14) with P(x, y) = 3x? + 6xy? and Q(x, y) = 6x?y + 4y3. 
Integrating P with respect to x and Q with respect to y and comparing results, we find that 
a potential function is given by the formula 
p(x, y) = x3 + 3x*y? + yt. 
By Theorem 10.7, each solution of the differential equation satisfies 
x3 + 3x2? + y? =C 


for some C. This provides an implicit representation of a family of integral curves. In 
this particular case the equation is quadratic in y? and can be solved to give an explicit 
formula for y in terms of x and C. 


EXAMPLE 2. Consider the first-order differential equation 


(10.15) ydx+2xdy=0. 
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Here P(x, y) = y and Q(x, y) = 2x. Since dP/dy = 1 and 0Q/dx = 2, this differential 
equation is not exact. However, if we multiply both sides by y we obtain an equation 
that is exact: 


(10.16) yr dx + Ixydy =0. 


A potential of the vector field y7f + 2xyj is p(x, y) = xy?, and every solution of (10.16) 
satisfies the relation xy? = C. This relation also represents a family of integral curves for 
Equation (10.15). 


The multiplier y which converted (10.15) into an exact equation 1s called an integrating 
factor. In general, if multiplication of a first-order linear equation by a nonzero factor 
L(x, y) results in an exact equation, the multiplier u(x, y) is called an integrating factor of 
the original equation. A differential equation may have more than one integrating factor. 
For example, u(x, y) = 2xy° is another integrating factor of (10.15). Some special differ- 
ential equations for which integrating factors can easily be found are discussed in the 
following set of exercises. 


10.20 Exercises 


Show that the differential equations in Exercises | through 5 are exact, and in each case find a 
one-parameter family of integral curves. 


. (x + 2y)dx + (2x + y)dy =0. 

. 2xydx + x®dy =0. 

. (x? — y)dx — (x + sin’ y) dy =0. 

. 4sin x sin 3y cos x dx — 3 cos 3y cos 2x dy = 0. 
. (3x?y + 8xy") dx + (x? + 8x?y + 12ye") dy =0. 


ON Mm Bb WN = 


. Show that a linear first-order equation, y’ + P(x)y = Q(x), has the integrating factor u(x) = 
elP(a)dx | Use this to solve the equation. 
7. Let u(x, y) be an integrating factor of the differential equation P(x, y) dx + Q(x, y)dy =0. 
Show that 
aP aQ a a 
er = O =, log |u| Pa ORs 


Use this equation to deduce the following rules for finding integrating factors: 

(a) If (@P/a@y — 2Q/0x)/Q is a function of x alone, say f(x), then e//'*)4= is an integrating 
factor. 

(b) If (eQ/ex — aP/dy)/P is a function of y alone, say g(y), then el9(v)dy is an integrating factor. 

8. Use Exercise 7 to find integrating factors for the following equations, and determine a one- 
parameter family of integral curves. 

(a) ydx — (2x + y)dy =0. 
(b) (x8 + y*®) dx — xy’ dy =0. 

9. If aP/dy — 0Q/ax = f(x)Q(x, y) — g(y)P(x, y), Show that exp {f f(x) dx + J g(y) dy} is an 
integrating factor of the differential equation P(x, y) dx + Q(x, y)dy =0. Find such an 
integrating factor for each of the following equations and obtain a one-parameter family of 
integral curves. 

(a) (2x*y + y?) dx + (2x° — xy) dy =0. 
(b) (e*secy — tany)dx + dy =0. 
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10. The following differential equations have an integrating factor in common. Find such an 
integrating factor and obtain a one-parameter family of integral curves for each equation. 


(3y + 4xy?) dx + (4x + 5x*y) dy =0, 
(6y + x*y?) dx + (8x + x*y) dy =0. 


10.21 Potential functions on convex sets 
In Theorem 10.6 we proved that the conditions 
Di f(x) = D; f(x) 


are necessary for a continuously differentiable vector field f= (/,,...,/,,) to be a gradient 
on an open set S in R". We then showed, by an example, that these conditions are not 
always sufficient. In this section we prove that the conditions are sufficient whenever the 
set Sis convex. The proof will make use of the following theorem concerning differentiation 
under the integral sign. 


THEOREM 10.8. Let S be a closed interval in R” with nonempty interior and let J = [a, b] 
be a closed interval in R'. Let Js, be the closed interval S x Jin R"*'. Write each point in 
Jina aS (x,t), wherexeSandteJ, 


(x,t) = (x,,...,X,,¢). 


Assume that p is a scalar field defined on J, such that the partial derivative D, is continuous 
on J,1,, where k is one of 1, 2,...,n. Define ascalar field y on S by the equation 


p(x) = | yx, pdt. 


Then the partial derivative D,,p exists at each interior point of S and is given by the formula 


D, g(x) = i D,,y(x, t) dt. 


In other words, we have 
D, |’ v(x, 1) dt = |” Dyylx, 0) dt. 


Note: This theorem is usually described by saying that we can differentiate under the 
integral sign. 


Proof. Choose any x in the interior of S. Since int Sis open, there is anr > 0 such that 


x + he, € int S for all h satisfying 0 < |A| <r. Here e, is the kth unit coordinate vector 
in R”. For such h we have 


(10.17) p(x + hey) — ox) = |” (ylx + he.) — lx, } dt. 
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Applying the mean value theorem to the integrand we have 


w(x + he,, t) _ v(x, t) =h Dw, t), 


where z lies on the line segment joining x and x + he,. Hence (10.17) becomes 


= b 
Therefore 
—_ b b 
eS = { D,.(x, t) dt = | {D,y(z, t) — Dyy(x, t)} dt. 


The last integral has absolute value not exceeding 


[ Deve, ) — Dyylx, 0] dt < (b — a) max |D, yz, 1) — Dy w(x, 0) 


where the maximum is taken for all z on the segment joining x to x + he,, and all ¢ in 
[a, b]. Now we invoke the uniform continuity of D, on S x J (Theorem 9.10) to conclude 
that for every « > 0 there is a 6 > 0 such that this maximum is <e/(b — a), whenever 
0 < |h| < 6. Therefore 


G(x + he,) — p(x) _ 


b 
5 | D,w(x, t) | <e whenever 0 < |h| <0. 


This proves that D,g(x) exists and equals |° D,p(x, t) dt, as required. 


Now we shall use this theorem to give the following necessary and sufficient condition for 
a vector field to be a gradient on a convex set. 


THEOREM 10.9. Let f= (fi,...,f,) be a continuously differentiable vector field on an 
open convex set Sin R". Then f is a gradient on S if and only if we have 


(10.18) D, f(x) = Di f(x) 
for each x in S and allk,j =1,2,...,n. 


Proof. We know, from Theorem 10.6, that the conditions are necessary. To prove 
sufficiency, we assume (10.18) and construct a potential m on S. 

For simplicity, we assume that S contains the origin. Let p(x) be the integral of f along 
the line segment from O to an arbitrary point x in S. As shown earlier in Equation (10.10) 
we have 


w(x) = JP flex) x dt = | yl, 0 at, 
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where p(x, ¢) = f(tx): x. There is a closed n-dimensional subinterval T of S with non- 
empty interior such that yp satisfies the hypotheses of Theorem 10.8 in T x J, where 


J = [0,1]. Therefore the partial derivative D,(x) exists for each k = 1, 2,..., n and 
can be computed by differentiating under the integral sign, 


1 
D,.9(x) = [, D,,y(x, t) dt. 
To compute D,y(x, t), we differentiate the dot product f(tx) - x and obtain 
Dyy(x, t) = f(tx)+ Dix + DL f(t} x 


= f(tx): e, + t(D, f(x), ..., Dif, (tx) ° x 
=f, (tx) + t(D, f,(tx), ..., Drf(tx)) > x, 


where in the last step we used the relation (10.18). Therefore we have 
D(x, t) = f,(tx) + VA (tx) +x. 
Now let g(t) = f,(tx). By the chain rule we have 
g(t) = VA(tx) * x 
so the last formula for D,y(x, t) becomes 
Dyyp(x, t) = g(t) + tg"(t). 

Integrating this from 0 to 1 we find 

(10.19) Dyplx) = [> Depts, t) = [° ge) at + | tg at. 


We integrate by parts in the last integral to obtain 


ik te'(t) dt = te(t) | — ii g(t) dt = g(1) — i g(t) dt. 


Therefore (10.19) becomes 
DyY(x) = gl) = f,(x). 


This shows that Vm = fon S, which completes the proof. 


I] 


MULTIPLE INTEGRALS 


11.1 Introduction 


Volume I discussed integrals J° f(x) dx, first for functions defined and bounded on finite 
intervals, and later for unbounded functions and infinite intervals. Chapter 10 of Volume 
II generalized the concept by introducing line integrals. This chapter extends the concept in 
yet another direction. The one-dimensional interval [a, b] is replaced by a two-dimensional 
set Q, called the region of integration. First we consider rectangular regions; later we 
consider more general regions with curvilinear boundaries. The integrand is a scalar field f 
defined and bounded on Q. The resulting integral is called a double integral and is denoted 
by the symbol 


ae: or by {] f(x, y) dx dy. 
Q Q 


As in the one-dimensional case, the symbols dx and dy play no role in the definition of the 
double integral; however, they are useful in computations and transformations of integrals. 

The program in this chapter consists of several stages. First we discuss the definition of 
the double integral. The approach here is analogous to the one-dimensional case treated in 
Volume I. The integral is defined first for step functions and then for more general 
functions. As in the one-dimensional case, the definition does not provide a useful 
procedure for actual computation of integrals. We shall find that most double integrals 
occurring in practice can be computed by repeated one-dimensional integration. We shall 
also find a connection between double integrals and line integrals. Applications of double 
integrals to problems involving area, volume, mass, center of mass, and related concepts 
are also given. Finally, we indicate how the concepts can be extended to n-space. 


11.2 Partitions of rectangles. Step functions 
Let Q be a rectangle, the Cartesian product of two closed intervals [a, b] and [c, d], 
Q = [a, 5] x [c,d] = {(x, y)|xe [a,b] and ye [e, d]}}. 


An example is shown in Figure 11.1. Consider two partitions P, and P, of [a, b] and [c, d], 
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Q = [a,b] x [c,d] 


FiGure 11.1 A rectangle Q, the Cartesian FiGure 11.2 A partition of a rectangle Q. 
product of two intervals. 


respectively, say 
Pie kigancneeae ey sand: Pees fe bs Vas 


where X9 = a, x, = 5, Vy =C, Ym =a. The Cartesian product P, x P, is said to be a 
partition of Q. Since P, decomposes [a, 5] into n subintervals and P, decomposes [c, d] into 
m subintervals, the partition P = P, X P, decomposes Q into mn subrectangles. Figure 
11.2 illustrates an example of a partition of Q into 30 subrectangles. A partition P’ of Q 
is said to be finer than P if P < P’, that is, if every point in P is also in P’. 

The Cartesian product of two open subintervals of P, and P, is a subrectangle with its 
edges missing. This is called an open subrectangle of P or of Q. 


DEFINITION OF STEP FUNCTION. A function f defined on a rectangle Q is said to be a step 
function if a partition P of Q exists such that f is constant on each of the open subrectangles 
of P. 


The graph of an example is shown in Figure 11.3. Most of the graph consists of horizontal 
rectangular patches. A step function also has well-defined values at each of the boundary 


Q 


Ficure 11.3 The graph of a step function defined on a rectangle Q. 
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points of the subrectangles, but the actual values at these points are not relevant to integra- 
tion theory. 

If f and g are two step functions defined on a given rectangle Q, then the linear combina- 
tion c, f + cog is also a step function. In fact, if P and P’ are partitions of Q such that fis 
constant on the open subrectangles of P and g is constant on the open subrectangles of P’, 
then c, f + c.g is constant on the open subrectangles of the union P U P’ (which we may 
call a common refinement of P and P’). Thus, the set of step functions defined on Q forms 
a linear space. 


11.3 The double integral of a step function 


Let P = P, X P, bea partition of a rectangle Q into mn subrectangles and let f be a step 
function that is constant on the open subrectangles of Q. Let the subrectangle determined 
by [x:;_1, x;] and [y,;_1, y,] be denoted by Q,, and let c,; denote the constant value that f 
takes at the interior points of Q;,. If fis positive, the volume of the rectangular box with 
base Q;,; and altitude c;,; is the product 


Cis ° (Xi — Xa); — Yj-1)- 


For any step function f, positive or not, the sum of all these products is defined to be the 
double integral of f over Q. Thus, we have the following definition. 


DEFINITION OF THE DOUBLE INTEGRAL OF A STEP FUNCTION. Let f be a step function which 
takes the constant value c;; on the open subrectangle (x;_,, Xi) X (yj-1, y;) of a rectangle Q. 
The double integral of f over Q is defined by the formula 


(11.1) in =, ») ») Cig (Xe — Xia) — Yi). 


Q 27=1 j=1 


As in the one-dimensional case, the value of the integral does not change if the partition 
P is replaced by any finer partition P’. Thus, the value of the integral is independent of the 
choice of P so long as f is constant on the open subrectangles of Q. 

For brevity, we sometimes write Ax; instead of (x; — x;_,) and Ay, instead of (y; — y;_1), 
and the sum in (11.1) becomes 


nem 
> 2 ¢is Ax; Ay;. 
To remind ourselves how this sum is formed, we write the symbol for the integral as 


izes y) dx dy. 
Q 


This symbol is merely an alternative notation for fff. 


Q 
Note that if f is constant on the interior of Q, say f(x, y) = k when a < x < 6b and 
c<y <d, we have 


(11.2) [| f= kb - ad — 0), 
Q 
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regardless of the values of f on the edges of Q. Since we have 


b—a=|' dx and d—c=|"ay, 


formula (11.2) can also be written as 


ay se ff Rrmndar= Leena] ae 
Q 


The integrals which appear on the right are one-dimensional integrals, and the formula 
is said to provide an evaluation of the double integral by repeated or iterated integration. 
In particular, when fis a step function of the type described above, we can write 


[hr=J? LE peamdx| ay=J® Lf" pen ay] ax 


Qii 


Summing on i and j and using (11.1), we find that (11.3) holds for step functions. 

The following further properties of the double integral of a step function are generaliza- 
tions of the corresponding one-dimensional theorems. They may be proved as direct 
consequences of the definition in (11.1) or by use of formula (11.3) and the companion 
theorems for one-dimensional integrals. In the following theorems the symbols s and f 
denote step functions defined on a rectangle Q. To avoid trivial special cases we assume 
that Q is a nondegenerate rectangle; in other words, that Q is not merely a single point 
or a line segment. 


THEOREM I1.1. LINEARITY PROPERTY. For every real c, and c, we have 


[J fers(x, ») + cate, yy] dx dy = ey | | s(x, y) dx dy + cy | f t(x, y) de ay. 
Q Q Q 


THEOREM 11.2. ADDITIVE PROPERTY. If Q is subdivided into two rectangles Q, and Qz, 
then 


ia s(x, y)dx dy = {| s(x, y)dx dy + [} scx, y)dx dy. 
Q Q1 Q2 
THEOREM 11.3 COMPARISON THEOREM. If s(x, y) < t(x, y) for every (x, y) in Q, we have 
[J sce, ») dx dy < Jf tx, y) dx ay. 
Q Q 
In particular, if t(x, y) > 0 for every (x, y) in Q, then 


[| «x, y) dx dy > 0. 
Q 


The proofs of these theorems are left as exercises. 
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11.4 The definition of the double integral of a function defined and bounded on a rectangle 


Let f be a function that is defined and bounded on a rectangle Q; specifically, suppose 
that 


Ifa yl<M if (x yeQ. 


Then f may be surrounded from above and from below by two constant step functions s 
and ¢, where s(x, y) = —M and ¢(x, y) = M for all (x, y) in Q. Now consider any two 
step functions s and ¢, defined on Q, such that 


(11.4) S(x,y) < f(x, vy) < U(x, y) for every point (x, y) in Q. 


DEFINITION OF THE INTEGRAL OF A BOUNDED FUNCTION OVER A RECTANGLE. /f there is one 
and only one number I such that 


(11.5) [[s<r<]fe 
Q Q 


for every pair of step functions satisfying the inequalities in (11.4), this number I is called 
the double integral of f over Q and is denoted by the symbol 


[fr or [| £0, ») dx dy. 
Q Q 
When such an I exists the function f is said to be integrable on Q. 


11.5 Upper and lower double integrals 


The definition of the double integral is entirely analogous to the one-dimensional case. 
Upper and lower double integrals can also be defined as was done in the one-dimensional 
case. 

Assume f is bounded on a rectangle Q and let s and ¢ be step functions satisfying (11.4). 
We say that s is below f, and t is above f, and we writes < f < t. Let S denote the set of all 
numbers {J s obtained as s runs through all step functions below /, and let T be the set of 


Q 
all numbers J{ ¢ obtained as ¢ runs through all step functions above f. Both sets S and T 
Q 
are nonempty since fis bounded. Also, ffs < {{tifs <f<t, so every number in S is 


Q Q 
less than every number in 7. Therefore S has a supremum, and 7 has an infimum, and they 
satisfy the inequalities 


[[s<sups<inf T < | Jt 
Q Q 


for all s and ¢ satisfying s << f< +t. This shows that both numbers sup S and inf 7 satisfy 
(11.5). Therefore, fis integrable on Q if and only if sup S = inf 7, in which case we have 


|| f= sup Ss = inf T. 
Q 
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The number sup S is called the /ower integral of f and is denoted by J(f/). The number 
inf T is called the upper integral of f and is denoted by /(f). Thus, we have 


i= sp|{s1s<J] nn =inc{[ fries, 
Q Q 
The foregoing argument proves the following theorem. 


THEOREM 11.4. Every function f which is bounded on a rectangle Q has a lower integral 
I(f) and an upper integral I(f) satisfying the inequalities 


[Issunsins|ft 
Q 


Q 


for all step functions s and t withs <f <1. The function f is integrable on Q if and only if 
its upper and lower integrals are equal, in which case we have 


[rary =i. 
Q 


Since the foregoing definitions are entirely analogous to the one-dimensional case, it is 
not surprising to learn that the linearity property, the additive property, and the comparison 
theorem as stated for step functions in Section 11.3, also hold for double integrals in general. 
The proofs of these statements are analogous to those in the one-dimensional case and will 
be omitted. 


11.6 Evaluation of a double integral by repeated one-dimensional integration 


In one-dimensional integration theory, the second fundamental theorem of calculus 
provides a practical method for calculating integrals. The next theorem accomplishes the 
same result in the two-dimensional theory; it enables us to evaluate certain double integrals 
by means of two successive one-dimensional integrations. The result is an extension of 
formula (11.3), which we have already proved for step functions. 


THEOREM 11.5. Let f be defined and bounded on a rectangle Q = [a, b] x [c,d], and 
assume that f is integrable on Q. For each fixed y in [c, d] assume that the one-dimensional 
integral |" f(x, y) dx exists, and denote its value by A(y). If the integral {¢ A(y) dy exists it 
is equal to the double integral {| f. In other words, we have the formula 

Q 


(11.6) [{ £0, ») ax dy = J" | P 4x, ») ax] ay. 


Q 


Proof. Choose any two step functions s and f satisfying s<f<¢ton Q. Integrating 
with respect to x over the interval [a, b] we have 


I’ s(x, y)dx < A(y) < ih t(x, y) dx. 
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Since the integral |¢ A(y) dy exists, we can integrate both these inequalities with respect to 
y over [c, d] and use Equation (11.3) to obtain 


|Js<[[aodys JJ 


Therefore {¢ A(y) dy is a number which lies between [fs and ff ¢ for all step functions s 


and f approximating f from below and from above, respectively. Since f is integrable on 
Q, the only number with this property is the double integral of f over Q. Therefore 
§2 A(y) dy = fff, which proves Equation (11.6). 

Q 


Formula (11.6) is said to provide an evaluation of the double integral by repeated or 
iterated integration. The process is described by saying that first we integrate f with respect 
to x from a to b (holding y fixed), and then we integrate the result with respect to y from 
cto d. If we interchange the order of integration, we have a similar formula, namely, 


(11.7) [[ Fe, y) dx ay = |? [[" Fe, yy dy] ax, 


Q 


which holds if we assume that {¢ f(x, y) dy exists for each fixed x in [a, b] and is integrable 
on [a, 5]. 


11.7 Geometric interpretation of the double integral as a volume 


Theorem 11.5 has a simple geometric interpretation, illustrated in Figure 11.4. If f is 
nonnegative, the set S of points (x, y, z) in 3-space with (x, y)inQand0 <z < f(x, y) Is 


Graph of f 


x The ordinate set S of fover QO =“ Cross section with area A(y)= De I(x, y) dx 
(a) (b) 


FiGuRE 11.4 The volume of S is the integral of the cross-sectional area: 
d 
(5) = [* A(y) dy. 
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called the ordinate set of f over Q. It consists of those points between the rectangle Q and 
the surface z = f(x, y). (See Figure 11.4(a).) For each y in the interval [c, d], the integral 


A(y) = ih F(x, y) dx 


is the area of the cross section of S cut by a plane parallel to the xz-plane (the shaded region 
in Figure 11.4(b)). Since the cross-sectional area A(y) is integrable on [c, d], Theorem 2.7 
of Volume I tells us that the integral {? A(y) dy is equal to v(S), the volume of S. Thus, for 
nonnegative integrands, Theorem 11.5 shows that the volume of the ordinate set of fover Q 
is equal to the double integral f{ /. 

Q 


Equation (11.7) gives another way of computing the volume of the ordinate set. This 
time we integrate the area of the cross sections cut by planes parallel to the yz-plane. 


11.8 Worked examples 


In this section we illustrate Theorem 11.5 with two numerical examples. 


EXAMPLE 1. If Q = [—1, 1] x [0, 7/2], evaluate {f(x sin y — ye*) dx dy, given that 


Q 
the integral exists. The region of integration is shown in Figure 11.5. 


Solution. Integrating first with respect to x and calling the result A(y), we have 


= —ey + yle. 


Pops ack ae r= 
A(y) = { (x sin y — ye*) dx = (5 sin y — ye) 
1 


a=—1 


2 2 
y>x 
m/2 
y=x 
yx? 
x x 
FiGureE 11.5 The region of integration for FIGURE 11.6 The region of integration for 


Example 1. Example 2. 
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Applying Theorem 11.5 we find 
: / 7/ 
[J (sin y — yet) dx dy = J"" AQ) ay = J"? (—ey + yle) dy 
Q 
7/2 P 
= (l/e — e) [ y dy = (l/e — e)n*/8. 


As a check on the calculations we may integrate first with respect to y: 


{| (x sin y — ye*)dx dy = i le (x sin y — ye") dy | dx 
Q 


y=7 /2 


= =|" (—x cos y — y*e*)| _ 
bs Po (— ne*/8 + x) dx = (lJe — e)n/8. 


EXAMPLE 2. If Q = [—1, 1] x [0, 2], evaluate the double integral iH Vly — x*| dx dy, 
given that it exists. 


Solution. If we integrate first with respect to y and call the result H(x), we have H(x) = 


q3 J ly — x?| dy. The region of integration is the rectangle shown in Figure 11.6. The 
parabola y = x? is also shown because of the presence of |y — x?| in the integrand. Above 
this parabola we have y > x? and below it we have y < x*. This suggests that we split the 
integral for H (x) as follows: 


2 ———————___— 2 —————— 2 —— 
H(x) = J Viy — #1 dy = f° Vx® — y dy + [avy — ay. 
We remember that x is treated as a constant in each of these integrals. In the first integral 


we make the change of variable t = x? — y and in the second we put ¢ = y — x*. This 
gives us 


H(x) = J? Viy — dy = —fovt ae t [P* Vit = ge + 32 — x, 


Applying Theorem 11.5 we find 
: 1 
{I Vly <—; x?| dx dy = | [2 x? + z(2 = x?)9/2] dx = t{ (2 _ xP? dx 
Q —1 0 


+ 


‘I 2)3/2 a 8 (= 
= =| x(2 — x°8? + 3x./2 — x + 6 arcsin =) 
3 2 


1 4 
3 


ud 


0 


The same result may be obtained by integrating first with respect to x, but the calculations 
are more complicated. 
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11.9 Exercises 


Evaluate the double integrals in Exercises | through 8 by repeated integration, given that each 
integral exists. 


1. ia xy(x + y)dxdy, where Q = [0,1] x [0, 1]. 
Q 

2. [| G2 +3x%y + y)dxdy, where @ = [0,1] x [0,1]. 
Q 

3. [| (Vy +x —3xy")dxdy, where Q = [0,1] x [1,3]. 
Q 

4. {| sin? x sin? y dx dy, where Q = [0, z] x [0, z]. 
Q 

5, {| sin (x + y)dxdy, where Q = [0, 2/2] x [0, 2/2]. 
Q 

6. || loos (x + yidxdy, where Q = [0,7] x [0, al. 
Q 

I, { f(x + y)dxdy, where Q = [0,2] x [0,2], and f(t) denotes the greatest integer < 1. 
Q 


= | [yen dx dy, where QO = [0,¢t] x [Il,¢t],¢ > 0. 
Q 


9. If Q is a rectangle, show that a double integral of the form j j f(x)gQ) dx dy is equal to the 


product of two one-dimensional integrals. State the assumptions about existence that you are 
using. 
10. Let f be defined on the rectangle Q = [0,1] x [0, 1] as follows: 


Lex = y if x+y<l, 


f(x,y) -| 


0 otherwise. 


Make a sketch of the ordinate set of fover Q and compute the volume of this ordinate set by 
double integration. (Assume the integral exists.) 


11. Solve Exercise 10 when 


x+y if x*®<y < 2%, 


f(*, y) -| 


0 otherwise. 


12. Solve Exercise 10 when Q = [-—1, 1] x [—1, 1] and 


xe+y? if xe+y? <1, 


f(x, y) -| 


0 otherwise. 
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13. Let f be defined on the rectangle Q = [1, 2] x [1, 4] as follows: 


(x+y? if x<y < 2x, 


f(x, y) -| 


0 otherwise. 


Indicate, by means of a sketch, the portion of Q in which fis nonzero and compute the value 
of the double integral {{ f, given that the integral exists. 


14. Let f be defined on the rectangle OQ = [0,1] x [0, 1] as follows: 


1 if x =y, 


fon =|, if x #y. 


Prove that the double integral {{ f exists and equals 0. 
Q 


11.10 Integrability of continuous functions 


The small-span theorem (Theorem 9.10) can be used to prove integrability of a function 
which is continuous on a rectangle. 


THEOREM 11.6. INTEGRABILITY OF CONTINUOUS FUNCTIONS. Jf a function f is continuous 
on a rectangle Q = [a,b] x [c,d], then f is integrable on Q. Moreover, the value of the 
integral can be obtained by iterated integration, 


(11.8) fh r=J7LP te,» ax] dy =)" [[* 70, y) dy] ax. 
Q 


Proof. Theorem 9.8 shows that f is bounded on Q, so f has an upper integral and a 
lower integral. We shall prove that J(f) = /(f). Choose « >0. By the small-span 
theorem, for this choice of ¢ there is a partition P of Q into a finite number (say n) of sub- 
rectangles Q,,..., Q, such that the span of fin every subrectangle is less than e«. Denote 
by M,(f) and m,(f), respectively, the absolute maximum and minimum values of / in 
Q,. Then we have 


M,(f) — m(f) < « 


foreach k = 1, 2,...,n. Now let s and ¢ be two step functions defined on the interior of 
each Q,, as follows: 


s(x) = m,(/), t(x) = M,(/) if xeintQ,. 
At the boundary points we define s(x) = mand t(x) = M, where mand M are, respectively, 


the absolute minimum and maximum values of fon Q. Then we have s < f < ¢ for all x 
in Q. Also, we have 


| | s= >) m(f)a(Q,) and {| t= y M,(f)a(Q,), 
Q k=1 Q k=1 
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where a(Q,) is the area of rectangle Q,. The difference of these two integrals is 
JJe— [Js =S (a — mN}aQ,) < «¥aQ,) = <a(Q), 
Q Q - fe 
where a(Q) is the area of Q. Since IWs< s<I<Ifh< IJ, we obtain the inequality 


0<1f) —I(f) < «<a(Q). 


Letting « > 0 we see that J/(f) = I(f), so fis integrable on Q. 

Next we prove that the double integral is equal to the first iterated integral in (11.8). 
For each fixed y in [c, d] the one-dimensional integral J° f(x, y) dx exists since the integrand 
is continuous on Q. Let A(y) = f° f(x, y) dx. We shall prove that A is continuous on 
[c,d]. If y and y, are any two points in [c, d] we have 


A(y) — AQ) = i (f(x, y) — f(x, yi} ax 


from which we find 
|A(y) — AQyi)| < (6 — a) max | f(x, y) — f(x, yl = (6 — a) If, y) —f(%, vol 


where x, is a point in [a, b] where | f(x, y) — f(x, y,)| attains its maximum. This inequality 
shows that A(y) — A(y,) as y—y,, so A is continuous at y,. Therefore the integral 
J? A(y) dy exists and, by Theorem 11.5, it is equal to J | f. A similar argument works when 
the iteration is taken in the reverse order. 


11.11 Integrability of bounded functions with discontinuities 


Let f be defined and bounded on a rectangle Q. In Theorem 11.6 we proved that the 
double integral of f over Q exists if fis continuous everywhere on Q. In this section we 
prove that the integral also exists if f has discontinuities in Q, provided the set of dis- 
continuities is not too large. To measure the size of the set of discontinuities we introduce 
the following concept. 


DEFINITION OF A BOUNDED SET OF CONTENT ZERO. Let A be a bounded subset of the plane. 
The set A is said to have content zero if for every « > 0 there is a finite set of rectangles whose 
union contains A and the sum of whose areas does not exceed «. 


In other words, a bounded plane set of content zero can be enclosed in a union of rec- 
tangles whose total area is arbitrarily small. 

The following statements about bounded sets of content zero are easy consequences of 
this definition. Proofs are left as exercises for the reader. 

(a) Any finite set of points in the plane has content zero. 

(b) The union of a finite number of bounded sets of content zero is also of content zero. 

(c) Every subset of a set of content zero has content zero. 

(d) Every line segment has content zero. 
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THEOREM 11.7. Let f be defined and bounded on a rectangle Q = [a, b] x [c,d]. If the 
set of discontinuities of f in Q is a set of content zero then the double integral \{ f exists. 
Q 


Proof. Let M> 0 be such that |f| < Mon Q. Let D denote the set of discontinuities 
of fin Q. Given 6 > 0, let P be a partition of Q such that the sum of the areas of all the 
subrectangles of P which contain points of D is less than 6. (This is possible since D has 
content zero.) On these subrectangles define step functions s and ¢ as follows: 


s(x) = —M, t((x)=M. 


On the remaining subrectangles of P define s and ¢ as was done in the proof of Theorem 11.6. 
Then we have s <<f<¢ throughout Q. By arguing as in the proof of Theorem 11.6 we 
obtain the inequality 


(11.9) [ft—J]s<ea(Q) + 2Mo. 
Q Q 


The first term, «a(Q), comes from estimating the integral of t — s over the subrectangles 
containing only points of continuity of f; the second term, 26, comes from estimating 
the integral of ¢ — s over the subrectangles which contain points of D. From (11.9) we 
obtain the inequality 


0< If) — If) < ea(Q) + 2M. 


Letting « > 0 we find 0 < J(f) —I(f) < 2M6. Since 6 is arbitrary this implies /(f) = 
I(f), so f is integrable on Q. 


11.12 Double integrals extended over more general regions 


Up to this point the double integral has been defined only for rectangular regions of 
integration. However, it is not difficult to extend the concept to more general regions. 

Let S be a bounded region, and enclose S in a rectangle Q. Let fbe defined and bounded 
on S. Define a new function f on Q as follows: 


f(x,y) if (,yeS, 
11.10 yy) = 
fey) f if (x,y)eQ—S. 


In other words, extend the definition of f to the whole rectangle Q by making the function 
values equal to 0 outside S. Now ask whether or not the extended function / is integrable 
on Q. If so, we say that fis integrable on S and that, by definition, 


[r= [Ir 
S Q 
First we consider sets of points S in the xy-plane described as follows: 


S={(x,y)|a<x<b and ox) <y< ¢,(x}, 
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where ¢, and ¢, are functions continuous on a closed interval [a, 5] and satisfying 9, < 2. 
An example of such a region, which we call a region of Type I, is shown in Figure 11.7. 
In a region of Type I, for each point f in [a, 5] the vertical line x = ¢ intersects S in a line 
segment joining the curve y = (x) to y = (x). Such a region is bounded because 9, 
and ¢» are continuous and hence bounded on [a, 5]. 

Another type of region T (Type II) can be described as follows: 


T={(x,y)|c<y<d and y,(y) <x < yy}, 


where y, and wy, are continuous on an interval [c, d] with y, < y.. An example is shown 
in Figure 11.8. In this case horizontal lines intersect TJ in line segments. Regions of Type 
II are also bounded. All the regions we shall consider are either of one of these two types or 
can be split into a finite number of pieces, each of which is of one of these two types. 


y 


FiGureE 11.7 A region S of Type I FiGurE 11.8 A region T of Type II. 


Let f be defined and bounded on a region S of Type I. Enclose S in a rectangle Q and 
define f on Q as indicated in Equation (11.10). The discontinuities of fin Q will consist of 
the discontinuities of fin S plus those points on the boundary of S at which fis nonzero. 
The boundary of S consists of the graph of 9, the graph of y,, and two vertical line 
segments joining these graphs. Each of the line segments has content zero. The next 
theorem shows that each of the graphs also has content zero. 


THEOREM 11.8. Let g be a real-valued function that is continuous on an interval [a, b]. 
Then the graph of y has content zero. 


Proof. Let A denote the graph of 9, that is, let 


= {(x,y)|y = v(x) and a<x <5}. 
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Choose any « > 0. We apply the small-span theorem (Theorem 3.13 of Volume I) to 
obtain a partition P of [a, 5] into a finite number of subintervals such that the span of » in 
every subinterval of P is less than e/(b — a). Therefore, above each subinterval of P the 
graph of lies inside a rectangle of height «/(b — a). Hence the entire of graph lies 
within a finite union of rectangles, the sum of whose areas is «. (An example is shown in 
Figure 11.9.) This proves that the graph of @ has content zero. 


FicureE 11.9 Proof that the graph of a continuous function » has content zero. 


The next theorem shows that the double integral J f fexists if fis continuous on int S, the 
interior of S. This is the set 


int S = {(x,y)|a<x<b and 9,(x)<y < g(x}. 


THEOREM 11.9. Let S be a region of Type 1, between the graphs of p, and y,. Assume that 
f is defined and bounded on S and that f is continuous on the interior of S. Then the double 
integral \\ f exists and can be evaluated by repeated one-dimensional integration, 
Ss 


(11.11) J) fs. y) dx dy = J” [[" fx, y) ay] dx. 


Proof. Let Q = [a, b] x [c, d] be a rectangle which contains S, and let f be defined by 
(11.10). The only possible points of discontinuity of fare the boundary points of S. Since 
the boundary of S has content zero, f is integrable on Q. For each fixed x in (a, b) the one- 
dimensional integral f¢ f(x, y) dy exists since the integrand has at most two discontinuities 
on [c,d]. Applying the version of Theorem 11.5 given by Equation (11.7) we have 


(11.12) [r= 2 L['7e, » ay] ax. 
Q 
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Now f(x, y) = f(x, y) if 9(x) < y < o(x), and f(x, y) = 0 for the remaining values of y 
in [c, d]. Therefore 


da 2a) 
[°70, 9) dy = J" fo, ») dy 
so Equation (11.12) implies (11.11). 


There is, of course, an analogous theorem for a region T of Type II. If fis defined and 
bounded on 7 and continuous on the interior of T, then fis integrable on 7 and the formula 
for repeated integration becomes 


(11.13) |} 46. y) dx dy = |" eee: f(x, y) dx] 


1(y) 


Some regions are of both Type I and Type II. (Regions bounded by circles and ellipses 
are examples.) In this case the order of integration is immaterial and we may write 


f° (Je? ses, ») dy] dx = [8 F009 ae] ay. 


In some cases one of these integrals may be much easier to compute than the other; it is 
usually worthwhile to examine both before attempting the actual evaluation of a double 
integral. 


11.13 Applications to area and volume 


Let S be a region of Type I given by 
S={(x,y)|a<x<b and p(X) <y< pv}. 


Applying Theorem 11.9 with f(x, y) = 1 for all (x, y) in S we obtain 
b 
i) dx dy = i [pa(x) — 91(x)] dx. 
s 


By Theorem 2.1 of Volume I, the integral on the right is equal to the area of the region S. 
Thus, double integrals can be used to compute areas. 

If f is nonnegative, the set of points (x, y, z) in 3-space such that (x, y)eS and 
0<z</f(x, y) is called the ordinate set of f over S. An example is shown in Figure 11.10. 
If fis nonnegative and continuous on S, the integral 


[* f(x, y) dy 


p(x) 


represents the area of a cross-section of the ordinate set cut by a plane parallel to the 
yz-plane (the shaded region in Figure 11.10). Formula (11.11) of Theorem 11.9 shows that 
the double integral of f over S is equal to the integral of the cross-sectional area. Hence 
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y 
: ?, (x) 
Area of cross section = ae f(x, y)dy 


Y= P2 (x) 
y=, (x) 


(a) ‘ 
FiGurRE 11.10 The integral {” ‘ f(x, y) dy is the area of a cross section of the ordinate 
1(x 


F ; b Po(x) 
set. The iterated integral | | { o 
a 


Gets f(, y) dy dx is the volume of the ordinate set. 
1 


the double integral {fis equal to the volume of the ordinate set of fover S. (See Theorem 
s 


2.7 of Volume I, p. 113.) 

More generally, if fand g are both continuous on S with f < g, then the double integral 
JJ (g —J) is equal to the volume of the solid lying between the graphs of the functions f 
Ss 


and g. Similar remarks apply, of course, to regions of Type II. 


11.14 Worked examples 


EXAMPLE 1. Compute the volume of the solid enclosed by the ellipsoid 


x2 yy? 7? 
a oF ¢ 


Solution. The solid in question lies between the graphs of two functions f and g, where 


g(x,y) =cV1 — x%/a? — y/b? and f(x, y) = —g(x, y). 


Here (x, y) € S, where S is the elliptical region given by 


S= {cx y) 


2 2 
4 test. 
a 


Applying Theorem 11.9 and using the symmetry we find that the volume V of the ellipsoidal 
solid is given by 


y=[fe-n=2fJe=s ft] 


(eee V1 — x2/a® — y?/b? dy | dx. 


0 
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Let A = V1 — x?/a®. Then the inner integral is 


|! Ja? = p68 dy = A V1 = yAby dy. 


«0 


Using the change of variable y = Ab sin t, dy = Ab cos t dt, we find that the last integral 
is equal to 


4 4 a 


7/2 2 
At | cost tdi = 7 At = 7 (1 — 2). 
0 


Therefore 


0 a 


a 2 
y = 8 (1-4) eee om 
4 3 


In the special case a = b = ¢ the solid is a sphere of radius a and the volume is $7ra°. 


EXAMPLE 2. The double integral of a positive function f, J{ f(x, y) dx dy, reduces to the 
repeated integral 8 


ih ie y) dy | dx. 


Determine the region S and interchange the order of integration. 


Solution. For each fixed x between 0 and 1, the integration with respect to y is over 
the interval from x? to x. This means that the region is of Type I and lies between the two 
curves y = x2 and y = x. The region S is the set of points between these two curves and 
above the interval [0, 1]. (See Figure 11.11.) Since S is also of Type II we may interchange 


FiGureE 11.11 Example 2. FiGuRE 11.12 Example 3. 
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the order of integration to obtain 
ip (vy 
PUD’ re, ») ax] ay. 


EXAMPLE 3. A double integral of a positive function f, J J f(x, y) dx dy, reduces to the 


repeated integral: 
ii Vihetes F(x, y) dx | dy. 


Determine the region S and interchange the order of integration. 


Solution. For each fixed y between 0 and 3, the integration with respect to x is over 


the interval from 4y/3 to /25 — y*. Therefore region S is of Type II and lies between 


the two curves x = 4y/3 and x = 25 — y*, This region, shown in Figure 11.12, is a 
sector of a circle. When the order of integration is reversed the region must be split into 
two regions of Type I; the result is the sum of two integrals: 


[Tf pon nay] ax + f° [ [ree » ay] a. 


11.15 Exercises 


In Exercises 1 through 5, make a sketch of the region of integration and evaluate the double 
integral. 


l. {] x cos (x + y) dx dy, where S is the triangular region whose vertices are (0, 0), (7, 0), (7, 7). 
Ss 


ae 


{| (1 + x) sin y dx dy, where S is the trapezoid with vertices (0,0), (1, 0), (1, 2), (0, 1). 
Ss 


oS) 


[| ett dx dy, where S = {(x, y) IIx + lyl <1}. 
S 


> 


| i x®y? dx dy, where S is the bounded portion of the first quadrant lying between the two 


iS 
hyperbolas xy = 1 and xy = 2 and the two straight lines y = x and y = 4x. 


ne {| (x? — y*) dx dy, where S is bounded by the curve y = sin x and the interval [0, 7]. 
Ss 


6. A pyramid is bounded by the three coordinate planes and the plane x + 2y + 3z = 6. Make 
a sketch of the solid and compute its volume by double integration. 

7. A solid is bounded by the surface z = x? — y?, the xy-plane, and the planes x = l and x = 3. 
Make a sketch of the solid and compute its volume by double integration. 

8. Compute, by double integration, the volume of the ordinate set of fover S if: 
(a) f(x,y) =x? + y? and 9S = {(x,y)| |x| <1, lyl < 1}. 
(b) f(x, y) =3x +y and S = {(x, y) | 4x? + 9y? < 36,x >0,y > O}. 
(c) f(x,y)=y+2x+20 and S=({(x,y)|x? + y? < 16}. 
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In Exercises 9 through 18 assume that the double integral of a positive function fextended over 
a region S reduces to the given repeated integral. In each case, make a sketch of the region S and 
interchange the order of integration. 


9 [oJ sen ax] ay. 14. Je] Pen dy] a. 

10. [iL Je Fes.y) as] ay Is. [LP ap See ay] ae 
u. [Lf fe] 6. [JE Sens ay] a. 

1 ff soa ae 17 POL tvcr S008] ax 
9 PL oS a] 18 LP fen ar] ty. 


19. When a double integral was set up for the volume V of the solid under the paraboloid z = x? + 
y® and above a region S of the xy-plane, the following sum of iterated integrals was obtained: 


y= [fffoe era] + PIP oe +e] a. 


Sketch the region S and express V as an iterated integral in which the order of integration is 
reversed. Also, carry out the integration and compute V. 

20. When a double integral was set up for the volume V of the solid under the surface z = f(x, y) 
and above a region S of the xy-plane, the following sum of iterated integrals was obtained: 


Gz (aay ee — fx, y)dx] dy + we | Py fe, y) dx | dy. 


asinc 
Given thatO <a <band0 <c < 7/2, sketch the region S, giving the equations of all curves 
which form its boundary. 


21. When a double integral was set up for the volume V of the solid under the surface z = f(x, y) 
and above a region S of the xy-plane, the following sum of iterated integrals was obtained: 


v= [LJP song] ae + [ILE sen» ay] a 


(a) Sketch the region S and express V as an iterated integral in which the order of integration 
is reversed. 
(b) Carry out the integration and compute V when f(x, y) = e*(x/ yy? 

22. Let A = fie" dt and B = J? e~“ dt. Evaluate the iterated integral 


1aaf' [ffera] 


in terms of A and B. These are positive integers m and vn such that 
= mA —nB +e! —e 1/4, 


Use this fact to check your answer. 
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23. A solid cone is obtained by connecting every point of a plane region S with a vertex not in the 
plane of S. Let A denote the area of S, and let A denote the altitude of the cone. Prove that: 
(a) The cross-sectional area cut by a plane parallel to the base and at a distance ¢ from the 
vertex is (t/h)?A, ifO0 <t <h. 
(b) The volume of the cone is }AA. 

24. Reverse the order of integration to derive the formula 


[oL[f em feo ax] dy = |} (a — seme fa) de, 


where a and m are constants, a > 0. 


11.16 Further applications of double integrals 


We have already seen that double integrals can be used to compute volumes of solids 
and areas of plane regions. Many other concepts such as mass, center of mass, and moment 
of inertia can be defined and computed with the aid of double integrals. This section 
contains a brief discussion of these topics. They are of special importance in physics and 
engineering. 

Let P denote the vector from the origin to an arbitrary point in 3-space. If positive 
masses m,, M2,..., Mm, are located at points P,, P,,..., P,, respectively, the center of 
mass Of the system is defined to be the point C determined by the vector equation 


C _ ae m,P,, ; 
Deni M 


The denominator, > m,, is called the total mass of the system. 
If each mass m, is translated by a given vector A to a new point Q, where Q, = P, + A, 
the center of mass is also translated by A, since we have 


> m,Q,, > m(P, + A) EL a a eae a 


This may also be described by saying that the location of the center of mass depends only 
on the points P,, P,,..., P, and the masses, and not on the origin. The center of mass is 
a theoretically computed quantity which represents, so to speak, a fictitious ‘“‘balance 
point’”’ of the system. 

If the masses lie in a plane at points with coordinates (x,, y,),...,(X,5),), and if the 
center of mass has coordinates (x, y), the vector relation which defines C can be expressed 
as two scalar equations 


ae > MyXj and pe > Myx 


> m, ym, 


In the numerator of the quotient defining x, the Ath term of the sum, m,x,, is called the 
moment of the mass m, about the y-axis. If a mass m equal to the total mass of the system 
were placed at the center of mass, its moment about the y-axis would be equal to the 
moment of the system, 


n 
mx => m,X;,. 
k=1 
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When we deal with a system whose total mass is distributed throughout some region 
in the plane rather than at a finite number of discrete points, the concepts of mass, center 
of mass, and moment are defined by means of integrals rather than sums. For example, 
consider a thin plate having the shape of a plane region S. Assume that matter is 
distributed over this plate with a known density (mass per unit area). By this we mean 
that there is a nonnegative function f defined on S and that f(x, y) represents the mass 
per unit area at the point (x, y). If the plate is made of a homogeneous material the density 
is constant. In this case the total mass of the plate is defined to be the product of the density 
and the area of the plate. 

When the density varies from point to point we use the double integral of the density as 
the definition of the total mass. In other words, if the density function fis integrable over 
S, we define the total mass m(S) of the plate by the equation 


m(S) = | f(x, y) dx dy. 
The quotient . 
LI FCs, y) dx dy 


mass _ 
area ff dx dy 
s 


is called the average density of the plate. If S is thought of as a geometric configuration 
rather than as a thin plate, this quotient is called the average or mean value of the function 
f over the region S. In this case we do not require f to be nonnegative. 

By analogy with the finite case, we define the center of mass of the plate to be the point 
(x, y) determined by the equations 


(11.14) xm(S) = {y xf (x, y) dx dy and ym(S) = {| yf(x, y) dx dy. 
s s 


The integrals on the right are called the moments of the plate about the y-axis and the 
X-axis, respectively. When the density is constant, say f(x, y) = c, a factor c cancels in 
each of Equations (11.14) and we obtain 


xa(S) = ii x dx dy and ya(S) = {{ ydxdy, 
s s 


where a(S) is the area of S. In this case the point (x, J) is called the centroid of the plate 
(or of the region S). 

If L is a line in the plane of the plate, let 6(x, y) denote the perpendicular distance from 
a point (x, y) in S to the line L. Then the number J, defined by the equation 


I, = | d°(x, y)f(x, y) dx dy 
S 


is called the moment of inertia of the plate about L. When f(x, y) = 1, J, is called the 
moment of inertia or second moment of the region S about L. The moments of inertia 
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about the x- and y-axes are denoted by /, and J,, respectively, and they are given by the 
integrals 


I= JJ yf ydedy and 1, =] [ x°f(x, y) dx dy. 
Ss Ss 
The sum of these two integrals is called the polar moment of inertia I, about the origin: 


Ip =I + 1, =| | (2 + yD F(x, y) dx dy. 
S 


Note: The mass and center of mass of a plate are properties of the body and are inde- 
pendent of the location of the origin and of the directions chosen for the coordinates axes. 
The polar moment of inertia depends on the location of the origin but not on the directions 
chosen for the axes. The moments and the moments of inertia about the x- and y-axes 
depend on the location of the origin and on the directions chosen for the axes. Ifa plate of 
constant density has an axis of symmetry, its centroid will lie on this axis. If there are two 
axes of symmetry, the centroid will lie on their intersection. These facts, which can be 
proven from the foregoing definitions, often help to simplify calculations involving center 
of mass and moment of inertia. 


EXAMPLE |. A thin plate of constant density c is bounded by two concentric circles with 
radii a and b and center at the origin, where 0 < b < a. Compute the polar moment of 
inertia. 


Solution. The integral for Jp is 


I, =e] [ (+ y*)axay, 
S 


where S = {(x, y) | b? < x? + y® < a®}. To simplify the computations we write the integral 
as a difference of two integrals, 


I= c {| (x? + y*)dx dy — e || (x? + y*) dx dy, 
S(a) S(o) 


where S(a) and S(d) are circular disks with radii a and b, respectively. We can use iterated 
integration to evaluate the integral over S(a), and we find 


a aa 4 
[forty yard =a [7] | (P+ dy | de 


0 
S(a) 


(We have omitted the details of the computation because this integral can be evaluated 
more easily with the use of polar coordinates, to be discussed in Section 11.27.) Therefore 
2 2 2 2 

h=>@ — b*) = xc(a® — py iF”) - moto 


bd 


where m = zrc(a? — b?), the mass of the plate. 
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EXAMPLE 2. Determine the centroid of the plane region bounded by one arch of a sine 
curve. 


Solution. We take the region S bounded by the curve y = sinx and the interval 
0<x<7a. Bysymmetry, the x-coordinate of the centroid is ¥ = 7/2. The y-coordinate, 
y, is given by 


J ydedy folsem*ydyldx  feadsin® x dx 


11.17 Two theorems of Pappus 


Pappus of Alexandria, who lived around 300 A.D., was one of the last geometers of the 
Alexandrian school of Greek mathematics. He wrote a compendium of eight books 
summarizing much of the mathematical knowledge of his time. (The last six and a part of 
the second are extant.) Pappus discovered a number of interesting properties of centroids, 
two of which are described in this section. The first relates the centroid of a plane region 
with the volume of the solid of revolution obtained by rotating the region about a line in 
its plane. 

Consider a plane region Q lying between the graphs of two continuous functions f and g 
over an interval [a,b], whereO << g <//f. Let S be the solid of revolution generated by 
rotating Q about the x-axis. Let a(Q) denote the area of Q, v(S) the volume of S, and jy the 
y-coordinate of the centroid of Q@. As Q is rotated to generate S, the centroid travels along a 
circle of radius y. Pappus’ theorem states that the volume of S is equal to the circumference 
of this circle multiplied by the area of Q; that 1s, 


(11.15) v(S) = 27ya(Q). 
To prove this formula we simply note that the volume is given by the integral 


u(S) = wf” [f%x) — 9(x)] dx 


and that j is given by the formula 


= b f(a) b 

ya(Q) =| J ydydx =|" | ? yay) ax =)" LPG) — 2) ax. 
Q 

Comparing these two formulas we immediately obtain (11.15). 

EXAMPLE |. Volume of a torus. Let S be the torus generated by rotating a circular disk Q 
of radius R about an axis at a distance b > R from the center of Q. The volume of S is 
easily calculated by Pappus’ theorem. We have y = b and a(Q) = 7R?, so 

v(S) = 27ya(Q) = 27? Rb. 


The next example shows that Pappus’ theorem can also be used to calculate centroids. 
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EXAMPLE 2. Centroid of a semicircular disk. Let y denote the y-coordinate of the centroid 
of the semicircular disk 


Q0={(x,y)|2@+y2< RR, y > O}. 


The area of Q is $7-R®. When Q is rotated about the x-axis it generates a solid sphere of 
volume $7-R°. By Pappus’ formula we have 


tr R = 2nj(GaR’), 


_ 4R 
BON ae 


The next theorem of Pappus states that the centroid of the union of two disjoint plane 
regions A and B lies on the line segment joining the centroid of A and the centroid of B. 
More generally, let A and B be two thin plates that are either disjoint or intersect in a set of 
content zero. Let m(A) and m(B) denote their masses and let C, and Cp denote vectors 
from an origin to their respective centers of mass. Then the union AU B has mass 
m(A) + m(B) and its center of mass is determined by the vector C, where 


= m(A)C4 + m(B)Cz 


(11.16) 
m(A) + m(B) 


The quotient for C is a linear combination of the form aC, + bCz, where a and b are 
nonnegative scalars with sum 1. A linear combination of this form is called a convex 
combination of C, and Cz. The endpoint of C lies on the line segment joining the endpoints 
of C, and Cp. 

Pappus’ formula (11.16) follows at once from the definition of center of mass given in 
(11.14). The proof is left as an exercise for the reader. The theorem can be extended in an 
obvious way to the union of three or more regions. It is especially useful in practice when a 
plate of constant density is made up of several pieces, each of which has geometric symmetry. 
We determine the centroid of each piece and then form a suitable convex combination to 
find the centroid of the union. Examples are given in Exercise 21 of the next section. 


11.18 Exercises 


In Exercises 1 through 8 a region S is bounded by one or more curves described by the given 
equations. In each case sketch the region S and determine the coordinates x and y of the centroid. 


y=x*, x+y =2. 

-y=axt3, ypou5 -x. 

x—-2y +8 =0, x+3y+5=0, x = -2, x=4, 
y =sin? x, y=0, O<x<z. 


y=sinx, y =cosx, O<x <7. 


. y =logx, y =0, Ly Gs 


x t+ Jy = 1, x=0, y=0. 
.x44y4=1, %x=0, y =0, in first quadrant. 


SNA vw PWNS 
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9. A thin plate is bounded by an arc of the parabola y = 2x — x? and the intervalO < x <2. 
Determine its mass if the density at each point (x, y) is (1 — y)/(l + x). 

10. Find the center of mass of a thin plate in the shape of a rectangle ABCD if the density at any 
point is the product of the distances of the point from two adjacent sides AB and AD. 


In Exercises 11 through 16, compute the moments of inertia J, and J, of a thin plate S in the 
xy-plane bounded by the one or more curves described by the given equations. In each case f(x, y) 
denotes the density at an arbitrary point (x, y) of S. 


Il. y =sin?'x, y = —sin’x, —r7<x<gnr; f(x,y) =1. 
See aa ae n = 
2.7 +5 =I, ee ee y = 0, 0<c <a, b>0; f(x,y) =1. 


13. (x -rP+(y—rr=r, «x =0, y=0, O<xK<r, OS<yKr; f(x,y) =1. 
l4.xy=1, xy=2, x=2y, y=2x, x >0, yod; (f(x,y) =1. 

15. y = e*, y=0, 0<x <a; f(x, y) = xy. 

16. y=J2x, y=0, O<x<2; f(x,y) =|x—-yl. 


17. Let S be a thin plate of mass m, and let Ly and L be two parallel lines in the plane of S, where 
Ly passes through the center of mass of S. Prove the parallel-axis theorem: 


Ir, ae a + mh? , 


where h is the perpendicular distance between the two lines L and Ly. [Hint: A careful choice 
of coordinates axes will simplify the work.] 

18. The boundary of a thin plate is an ellipse with semiaxes a and b. Let L denote a line in the 
plane of the plate passing through the center of the ellipse and making an angle « with the axis 
of length 2a. If the density is constant and if the mass of the plate is m, show that the moment 
of inertia J; is equal to gm(a? sin? « + b® cos® «). 

19. Find the average distance from one corner of a square of side / to points inside the square. 

20. Let 6 denote the distance from an arbitrary point P inside a circle of radius r to a fixed point 
Py whose distance from the center of the circle is h. Find the average of the function 6? over 
the region enclosed by the circle. 

21. Let A, B, C denote the following rectangles in the xy-plane: 


A=[0,4] x [0,1], B=[2,3)xf1,3], C=[,4] x [3,4]. 


Use a theorem of Pappus to determine the centroid of each of the following figures: 
(a) AUB. (c) BUC. 
(b) AUC. (d) AUBUC. 

22. An isosceles triangle T has base 1 and altitude 4. The base of T coincides with one edge of a 
rectangle R of base 1 and altitude 2. Find a value of h so that the centroid of R U T will lie 
on the edge common to R and T. 

23. An isosceles triangle T has base 2r and altitude 4. The base of T coincides with one edge of a 
semicircular disk D of radius r. Determine the relation that must hold between r and / so that 
the centroid of T U D will lie inside the triangle. 


11.19 Green’s theorem in the plane 


The second fundamental theorem of calculus for line integrals states that the line integral 
of a gradient Vf along a path joining two points a and b may be expressed in terms of the 
function values f(a) and f(b). There is a two-dimensional analog of the second fundamental 
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theorem which expresses a double integral over a plane region R as a line integral taken 
along a closed curve forming the boundary of R. This theorem is usually referred to as 
Green’s theorem.} It can be stated in several ways; the most common is in the form of the 
identity : 


(11.17) { (2-2) ax dy = P dx + Qdy. 
a Ox Oy C 


The curve C which appears on the right is the boundary of the region R, and the integra- 
tion symbol 4 indicates that the curve is to be traversed in the counterclockwise direction, 


as suggested by the example shown in Figure 11.13. 
—_——~ 


Ficure 11.13 The curve C is the boundary of R, traversed in a counterclockwise 
direction. 


Two types of assumptions are required for the validity of this identity. First, conditions 
are imposed on the functions P and @Q to ensure the existence of the integrals. The usual 
assumptions are that P and Q are continuously differentiable on an open set S containing 
the region R. This implies continuity of P and Q on C as well as continuity of dP/dy and 
0Q/0x on R, although the theorem is also valid under less stringent hypotheses. Second, 
there are conditions of a geometric nature that are imposed on the region K and its boundary 
curve C. The curve C may be any rectifiable simple closed curve. The term “‘rectifiable”’ 
means, of course, that C has a finite arc length. To explain what is meant by a simple 
closed curve, we refer to the vector-valued function which describes the curve. 

Suppose C is described by a continuous vector-valued function « defined on an interval 
[a, b]. If a(a) = a(b), the curve is closed. A closed curve such that a(t,) ¥ a(t.) for every 
pair of values ¢, ¥ ¢, in the half-open interval (a, b] is called a simple closed curve. This 
means that, except for the end points of the interval [a, 5], distinct values of ¢ lead to 
distinct points on the curve. A circle is the prototype of a simple closed curve. 

Simple closed curves that lie in a plane are usually called Jordan curves in honor of 
Camille Jordan (1838-1922), a famous French mathematician who did much of the pioneer- 
ing work on such concepts as simple closed curves and arc length. Every Jordan curve C 


+ In honor of George Green (1793-1841), an English mathematician who wrote on the applications of 
mathematics to electricity and magnetism, fluid flow, and the reflection and refraction of light and sound. 
The theorem which bears Green’s name appeared earlier in the researches of Gauss and Lagrange. 
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decomposes the plane into two disjoint open connected sets having the curve C as their 
common boundary. One of these regions is bounded and 1s called the interior (or inner 
region) of C. (An example is the shaded region in Figure 11.13.) The other is unbounded 
and is called the exterior (or outer region) of C. For some familiar Jordan curves such as 
circles, ellipses, or elementary polygons, it is intuitively evident that the curve divides 
the plane into an inner and an outer region, but to prove that this is true for an arbitrary 
Jordan curve is not easy. Jordan was the first to point out that this statement requires 
proof; the result is now known as the Jordan curve theorem. Toward the end of the 19th 
century Jordan and others published incomplete proofs. In 1905 the American mathe- 
matician Oswald Veblen (1880-1960) gave the first complete proof of this theorem. 
Green’s theorem is valid whenever C is a rectifiable Jordan curve, and the region R is the 
union of C and its interior.t Since we have not defined line integrals along arbitrary 
rectifiable curves, we restrict our discussion here to piecewise smooth curves. 

There is another technical difficulty associated with the formulation of Green’s theorem. 
We have already remarked that, for the validity of the identity in (11.17), the curve C 
must be traversed in the counterclockwise direction. Intuitively, this means that a man 
walking along the curve in this direction always has the region R to his left. Again, for 
some familiar Jordan curves, such as those mentioned earlier, the meaning of the expression 
“traversing a curve in the counterclockwise direction” is intuitively evident. However, in a 
strictly rigorous treatment of Green’s theorem one would have to define this expression in 
completely analytic terms, that is, in terms of the vector-valued function @ that describes 
the curve. One possible definition is outlined in Section 11.24. 

Having pointed out some of the difficulties associated with the formulation of Green’s 
theorem, we shall state the theorem in a rather general form and then indicate briefly why 
it is true for certain special regions. In this discussion the meaning of ‘‘counterclockwise’”’ 
will be intuitive, so the treatment is not completely rigorous. 


THEOREM 11.10. GREEN’S THEOREM FOR PLANE REGIONS BOUNDED BY PIECEWISE SMOOTH 
JORDAN CURVES. Let P and Q be scalar fields that are continuously differentiable on an 
open set S in the xy-plane. Let C be a piecewise smooth Jordan curve, and let R denote the 
union of C and its interior. Assume R is a subset of S. Then we have the identity 


0Q _ oP =4 
(11.18) {| ie ~s dx dy anes + Qdy, 
R 


where the line integral is taken around C in the counterclockwise direction. 


Note: The identity in (11.18) is equivalent to the two formulas 


A dy = d 
(11.19) iE x y= 0 ly 


R 


+ A proof of Green’s theorem for regions of this generality can be found in Chapter 10 of the author’s 
Mathematical Analysis. 
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and 


(11.20) “Ree Pads. 
C 


In fact, if both of these are true, (11.18) follows by addition. Conversely, if (11.18) is true 
we may obtain (11.19) and (11.20) as special cases by taking P = Oand Q = 0, respectively. 


Proof for special regions. We shall prove (11.20) for a region R of Type I. Such a region 
has the form 


={(x,y)|a<x<b and f(x) <y < g(x}, 
where f and g are continuous on [a, b] with f < g. The boundary C of R consists of four 


parts, a lower arc C, (the graph of f), an upper arc C, (the graph of g), and two vertical line 
segments, traversed in the directions indicated in Figure 11.14. 


FiGure 11.14 Proof of Green’s theorem for a FiGure 11.15 Proof of Green’s theorem for 
special region. a more general region. 


First we evaluate the double integral — J. J (@P/dy) dx dy by iterated integration. Integrat- 
ing first with respect to y we have 


(11.21) | hi dx dy = ~{"| oP ay | dx =| | = iy dx 
. Oy f(x) Oy a g(x) Oy 
= |" Px, foo) dx — |” Phx, (x)] dx. 


On the other hand, the line integral {~ P dx can be written as follows: 


[,Pax= |] Paxt |, Pax, 


since the line integral along each vertical segment is zero. To evaluate the integral along 
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C, we use the vector representation a(t) = ti + f(¢)j and obtain 


ie Pdx = i Pt, f(t)] dt. 


To evaluate the integral along C, we use the representation a(t) = ti + g(t)j and take into 
account the reversal in direction to obtain 


[..? dx = =) P[t, g(t) dt. 


Therefore we have 


[ Pax =|" Pt selae— |” Pte, eo) at. 


Comparing this equation with the formula in (11.21) we obtain (11.20). 

A similar argument can be used to prove (11.19) for regions of Type II. In this way a 
proof of Green’s theorem is obtained for regions that are of both Type I and Type II. 
Once this is done, the theorem can be proved for those regions R that can be decomposed 
into a finite number of regions that are of both types. “‘Crosscuts”’ are introduced as shown 
in Figure 11.15, the theorem is applied to each subregion, and the results are added 
together. The line integrals along the crosscuts cancel in pairs, as suggested in the figure, 
and the sum of the line integrals along the boundaries of the subregions is equal to the line 
integral along the boundary of R. 


11.20 Some applications of Green’s theorem 
The following examples illustrate some applications of Green’s theorem. 
EXAMPLE 1. Use Green’s theorem to compute the work done by the force field 


f(x, y) = (vy + 3x)i + (2y — x)jin moving a particle once around the ellipse 4x? + y? = 4 
in the counterclockwise direction. 


Solution. The work is equal to [~ Pdx + Ody, where P = y + 3x, Q = 2y — x, and 
C is the ellipse. Since 0Q/0x — OP/dy = —2, Green’s theorem gives us 


[_P AOA 2 JI (Dias oaR): 


where a(R) is the area of the region enclosed by the ellipse. Since this ellipse has semiaxes 
a = | and b = 2, its area is mab = 27 and the value of the line integral is —47. 


EXAMPLE 2. Evaluate the line integral J, (5 — xy — y®) dx — (2xy — x®) dy, where Cis 
the square with vertices (0, 0), (1, 0), (1, 1), (0, 1), traversed counterclockwise. 


Solution. Here P= 5 — xy — y?, Q = x? — 2xy, and 0Q/dx — OP/dy = 3x. Hence, 
by Green’s theorem, we have 


[Pax +Qdy =3]| xdxdy =32, 
R 
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where x is the x-coordinate of the centroid of the square. Since x is obviously 3, the value 
of the line integral is 3. 


EXAMPLE 3. Area expressed as a line integral. The double integral for the area a(R) of a 
region R can be expressed in the form 


ay= J dvay= Jf (2-3) oe 


where P and Q are such that 0Q/0x — OP/dy = 1. For example, we can take Q(x, y) = 4x 
and P(x, y) = —4y. If Ris the region enclosed by a Jordan curve C we can apply Green’s 
theorem to express a(R) as a line integral, 


a(R) = | Pdx +Qdy= 5] — ydx+xdy. 
C 2Jc 


If the boundary curve C is described by parametric equations, say 
x=X(t), y= Y(t), a<t<b, 


the line integral for area becomes 


X(t) Y(t) 
X'(t) Y(t) 


b b 
a(R) => | {-YOX'H + XOY'O} dt =5 | 


dt. 
2 


11.21 A necessary and sufficient condition for a two-dimensional vector field to be a 
gradient 


Let f(x, y) = P(x, y)i + Q(x, y)j be a vector field that is continuously differentiable on 
an open set S in the plane. If fis a gradient on S we have 


(11.22) seg 
Oy Ox 


everywhere on S. In other words, the condition (11.22) is necessary for f to be a gradient. 
As we have already noted, this condition is not sufficient. For example, the vector field 


eee 
x? + y? 


LO ra J 
satisfies (11.22) everywhere on the set S = R? — {(0,0)}, but fis not a gradient on S. 
In Theorem 10.9 we proved that condition (11.22) is both necessary and sufficient for f 
to be a gradient on S if the set S is convex. With the help of Green’s theorem we can extend 
this result to a more general class of plane sets known as simply connected sets. They are 
defined as follows. 
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DEFINITION OF A SIMPLY CONNECTED PLANE SET. Let S be an open connected set in the 
plane. Then S is called simply connected if, for every Jordan curve C which lies in S, the inner 
region of C is also a subset of S. 


An annulus (the set of points lying between two concentric circles) is not simply connected 
because the inner region of a circle concentric with the bounding circles and of radius 
between theirs is not a subset of the annulus. Intuitively speaking, we say that S is simply 
connected when it has no “holes.’” Another way to describe simple connectedness is to say 
that a curve C, in S connecting any two points may be continuously deformed into any 
other curve C, in S joining these two points, with all intermediate curves during the 
deformation lying completely in S. An alternative definition, which can be shown to be 
equivalent to the one given here, states that an open connected set S is simply connected if 
its complement (relative to the whole plane) is connected. For example, an annulus is not 
simply connected because its complement is disconnected. An open connected set that is 
not simply connected is called multiply connected. 


THEOREM I1.11. Let f(x, y) = P(x, y)i + Q(x, y)j be a vector field that is continuously 
differentiable on an open simply connected set S in the plane. Then f is a gradient on S if and 
only if we have 


(11.23) —=— everywhere onS. 


Proof. As already noted, condition (11.23) is necessary for fto be a gradient. We shall 
prove now that it is also sufficient. 

It can be shown that in any open connected plane set S, every pair of points a and x 
can be joined by a simple step-polygon, that is, by a polygon whose edges are parallel to 
the coordinate axes and which has no self-intersections. If the line integral of ffrom a to x 
has the same value for every simple step-polygon in S joining a to x, then exactly the same 
argument used to prove Theorem 10.4 shows that fis a gradient on S. Therefore, we need 


FiGurReE 11.16 Independence of the path in a simply connected region. 


Exercises 385 


only verify that the line integral of f from a to x has the same value for every simple step- 
polygon in S joining a to x. 

Let C, and C, be two simple step-polygons in S joining ato x. Portions of these polygons 
may coincide along certain line segments. The remaining portions will intersect at most a 
finite number of times, and will form the boundaries of a finite number of polygonal 
regions, say R,,..., R,,. Since S is assumed to be simply connected, each of the regions 
R, is a subset of S. An example is shown in Figure 11.16. The solid line represents C,, 
the dotted line represents C,, and the shaded regions represent R,,..., R,,. (These two 
particular polygons coincide along the segment pq.) 

We observe next that the line integral of f from a to x along C, plus the integral from x 
back to a along C, is zero because the integral along the closed path is a sum of integrals 
taken over the line segments common to C, and C, plus a sum of integrals taken around 
the boundaries of the regions R,. The integrals over the common segments cancel in pairs, 
since each common segment is traversed twice, in opposite directions, and their sum Is 
zero. The integral over the boundary I’, of each region R,, is also zero because, by Green’s 


theorem we may write 
[Pax + Ody = + |{(2-2) dx dy, 
= Ox oy 


Ty 


and the integrand of the double integral is zero because of the hypothesis 0Q/0x = dP/dy. 
It follows that the integral from a to x along C, is equal to that along C,. As we have 
already noted, this implies that fis a gradient in S. 


11.22 Exercises 


1. Use Green’s theorem to evaluate the line integral $, y* dx + x dy when 
(a) Cis the square with vertices (0, 0), (2, 0), (2, 2), (0, 2). 
(b) Cis the square with vertices (+1, +1). 
(c) Cis the square with vertices (+2, 0), (0, +2). 
(d) C is the circle of radius 2 and center at the origin. 
(e) Chas the vector equation a(t) = 2 cos? ti + 2sin’tj7,0 <t < 27. 


2. If P(x, y) = xe” and Q(x, y) = —x®y ev" + 1/(x? + y’), evaluate the line integral $ Pdx + 
Q dy around the boundary of the square of side 2a determined by the inequalities |x| < a@ and 


yl <a. 
3. Let C be a simple closed curve in the xy-plane and let J, denote the moment of inertia (about 
the z-axis) of the region enclosed by C. Show that an integer n exists such that 


nl, = $, x* dy — yr dx. 


4. Given two scalar fields uv and v that are continuously differentiable on an open set containing 
the circular disk R whose boundary is the circle x? + y? = 1. Define two vector fields fand g 
as follows: 

; ; Ou Ou . dv dv\. 
f(x,y) = v(x, yi tu, yi, By) = & = i ai (= = 5) 
Find the value of the double integral {f f- g dx dy if it is known that on the boundary of R we 
have u(x, y) = landu(x,y)=y. * 
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5. If fand g are continuously differentiable in an open connected set S in the plane, show that 
. [Vg :da= -$, &V f: da for every piecewise smooth Jordan curve C in S. 


6. Let u and v be scalar fields having continuous first- and second-order partial derivatives in an 
open connected set S in the plane. Let R be a region in S bounded by a piecewise smooth Jordan 
curve C. Show that: 


Ou Ou Ov Ov 
(a) $ dx + uvdy = {[{ (5 _ 4 + (= _ 1) dx dy. 
R 
l Ou dv av a2y 
om 5% ("a a) ix + (uz - 05) o - | (+a ax ay” Ox 5) a4. 


Normal derivatives. In Section 10.7 we defined line integrals with respect to arc length in such a 
way that the following equation holds: 


| Pax + Qdy= [f- T ds, 


where f = Pi + Qjand T is the unit tangent.vector to C. (The dot product f- T is called the 
tangential component of f along C.) If C is a Jordan curve described by a continuously differ- 
entiable function a, say a(t) = X(t)i + Y(t)j, the unit outer normal n of C is defined by the 
equation 


n(t) = (Y'(t)i — X'(1)) 


e"@II a) 


whenever ||«’(t)|| #0. If y is a scalar field with gradient V¢ on C, the normal derivative a9/ én is 
defined on C by the equation 


This is, of course, the directional derivative of ¢ in the direction of m. These concepts occur in the 
remaining exercises of this section. 


7. If f = Qi — Pj, show that 
[ Pax +Ody=| finds. 


(The dot product f: n is called the normal component of falong C.) 

8. Let f and g be scalar fields with continuous first- and second-order partial derivatives on an 
open set S in the plane. Let R denote a region (in S) whose boundary is a piecewise smooth 
Jordan curve C. Prove the following identities, where V2u = 0?u/0x? + d?u/dy*. 


8 
(a) —ds = V29 dx dy. 
co on - 


) 
(b) Zw a (fV2g + Uf: Vg) dx dy. 


(c) $(re ~g v ds =| (f Vg — gV2f) dx dy. 
R 
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The identity in (c) is known as Green’s formula; it shows that 


dg of 
$ ea -> gz ds 


whenever f and g¢ are both harmonic on R (that is, when V?f = V2¢ = 0 on R). 
9. Suppose the differential equation 


P(x, y) dx + Q(x, y)dy =0 


has an integrating factor u(x, y) which leads to a one-parameter family of solutions of the form 
p(x, y) = C. Ifthe slope of the curve g(x, y) = C at (x, y) is tan 6, the unit normal vector z is 
taken to mean 


n= sin Oi —cos 0/. 


There is a scalar field ¢(x, y) such that the normal derivative of is given by the formula 


op 
On: a U(x, yg (x, y) ) 
where 0g/0n = Voy-n. Find an explicit formula for g(x, y) in terms of P(x, y) and Q(x, y). 


% 11.23 Green’s theorem for multiply connected regions 


Green’s theorem can be generalized to apply to certain multiply connected regions. 


THEOREM 11.12. GREEN’S THEOREM FOR MULTIPLY CONNECTED REGIONS. Let C,,...,C, 
be n piecewise smooth Jordan curves having the following properties: 

(a) No two of the curves intersect. 

(b) The curves C,,..., C,, all lie in the interior of C,. 

(c) Curve C; lies in the exterior of curve C; for eachiAj,i>1,j> 1. 
Let R denote the region which consists of the union of C, with that portion of the interior of C, 
that is not inside any of the curves C,, C3, ..., Cy. (An example of such a region is shown 
in Figure 11.17.) Let P and Q be continuously differentiable on an open set S containing R. 
Then we have the following identity: 


P 


(11.24) i) ees Rip $ (dx + 0d) -¥ Prax +d), 


The theorem can be proved by introducing crosscuts which transform R into a union 
of a finite number of simply connected regions bounded by Jordan curves. Green’s theorem 
is applied to each part separately, and the results are added together. We shall illustrate 
how this proof may be carried out when n = 2. The more general case may be dealt with 
by using induction on the number v of curves. 

The idea of the proof when n = 2 is illustrated by the example shown in Figure 11.18, 
where C, and C, are two circles, C, being the larger circle. Introduce the crosscuts AB 
and CD, as shown in the figure. Let K, denote the Jordan curve consisting of the upper 
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FicureE 11.17 A multiply connected region. Ficure 11.18 Proof of Green’s theorem 
for a multiply connected region. 


half of C,, the upper half of C,, and the segments AB and CD. Let K, denote the Jordan 
curve consisting of the lower half of C,, the lower half of C,, and the two segments AB 
and CD. Now apply Green’s theorem to each of the regions bounded by K, and K, and 
add the two identities so obtained. The line integrals along the crosscuts cancel (since each 
crosscut is traversed once in each direction), resulting in the equation 


The minus sign appears because of the direction in which C, is traversed. This is Equation 
(11.24) when n = 2. 


For a simply connected region, the condition ¢P/dy = 0Q/dx implies that the line 
integral { Pdx + Q dy is independent of the path (Theorem 11.11). As we have already 
noted, if S is not simply connected, the condition dP/dy = 0Q/dx does not necessarily 
imply independence of the path. However, in this case there is a substitute for inde- 
pendence that can be deduced from Theorem 11.12. 


THEOREM 11.13. INVARIANCE OF A LINE INTEGRAL UNDER DEFORMATION OF THE PATH. 
Let P and Q be continuously differentiable on an open connected set S in the plane, and assume 
that OP/dy = 0Q/0x everywhere on S. Let C, and C, be two piecewise smooth Jordan curves 
lying in S and satisfying the following conditions: 

(a) C, lies in the interior of C,. 

(b) Those points inside C, which lie outside C, are in S. (Figure 11.19 shows an example.) 
Then we have 


(11.25) $. Pdx+Qdy=$ Pdx+Qay, 
where both curves are traversed in the same direction. 


Proof. Under the conditions stated, Equation (11.24) is applicable when n = 2. The 
region R consists of those points lying between the two curves C, and C, and the curves 
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FicureE 11.19. Invariance of a line integral under deformation of the path. 


themselves. Since dP/dy = 0Q/0x in S, the left member of Equation (11.24) is zero and 
we obtain (11.25). 


Theorem 11.13 is sometimes described by saying that if 0P/dy = 0Q/dx in S the value of 
a line integral along a piecewise smooth simple closed curve in S is unaltered if the path is 
deformed to any other piecewise smooth simple closed curve in S, provided all intermediate 
curves remain within the set S during the deformation. The set S is assumed to be open 
and connected—it need not be simply connected. 


%11.24 The winding number 


We have seen that the value of a line integral often depends both on the curve along 
which the integration takes place and on the direction in which the curve is traversed. 
For example, the identity in Green’s theorem requires the line integral to be taken in the 
counterclockwise direction. In a completely rigorous treatment of Green’s theorem it 
would be necessary to describe analytically what it means to traverse a closed curve in 
the “counterclockwise direction.’’ For some special curves this can be done by making 
specific statements about the vector-valued function « which describes the curve. For 
example, the vector-valued function @ defined on the interval [0, 277] by the equation 


(11.26) a(t) = (acost + X9)i + (asin t + yo)jJ 


describes a circle of radius a with center at (x9, yo). This particular function is said to 
describe the circle in a positive or counterclockwise direction. On the other hand, if we 
replace t by —f on the right of (11.26) we obtain a new function which is said to describe 
the circle in a negative or clockwise direction. In this way we have given a completely 
analytical description of ‘‘clockwise’’ and “‘counterclockwise”’ for a circle. However, it 
is not so simple to describe the same idea for an arbitrary closed curve. For piecewise 
smooth curves this may be done by introducing the concept of the winding number, an 
analytic device which gives us a mathematically precise way of counting the number of 
times a radius vector @ “winds around” a given point as it traces out a given closed curve. 
In this section we shall describe briefly one method for introducing the winding number. 
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Then we shall indicate how it can be used to assign positive and negative directions to 
closed curves. 

Let C be a piecewise smooth closed curve in the plane described by a vector-valued 
function @ defined on an interval [a, b], say 


a(‘)=X(ti+ Yj iif act<b. 


Let Py = (Xo, Yo) be a point which does not lie on the curve C. Then the winding number 
of a with respect to the point P, is denoted by W(a; P); it is defined to be the value of the 
following integral: 


(1! 27) W(a; P ) = Af [ X(t) a Xol Y(t) az [Y(t) = Vol X(t) dt 
| Oma (XO) = xe? + [YO = yok? 


This is the same as the line integral 


(11.28) i $ a6 ie 1 oh ees 
2Qrdc 8 =—(X — Xo)” + (y — Yo)? 


It can be shown that the value of this integral is always an integer, positive, negative, or 
zero. Moreover, if C is a Jordan curve (simple closed curve) this integer is 0 if Py is outside 
C and has the value +1 or —1 if Py is inside C. (See Figure 11.20.) Furthermore, W(a; Py) 


e Po 


—7 
Winding number + | Winding number — | Winding number 0 


Ficure 11.20 Illustrating the possible values of the winding number of a Jordan 
curve C with respect to Pp. 


is either +1 for every point Py inside C or it is —1 for every such point. This enables us 
to define positive and negative orientations for C as follows: If the winding number 
W (a; Po) is +1 for every point P, inside C we say that @ traces out C in the positive or 
counterclockwise direction. \f the winding number is —1 we say that @ traces out C in 
the negative or clockwise direction. [An example of the integral in (11.28) with x» = yp = 0 
was encountered earlier in Example 2 of Section 10.16.] 

To prove that the integral for the winding number is always +1 or —1 for a simple 
closed curve enclosing (Xo, Yo) we use Theorem 11.13. Let S denote the open connected 
region consisting of all points in the plane except (Xo, yo). Then the line integral in (11.28) 
may be written as J; Pdx + Qdy, and it is easy to verify that 0P/dy = 0Q/0x every- 
where in S. Therefore, if (x9, yo) 1s inside C, Theorem 11.13 tells us that we may replace 
the curve C by a circle with center at (x9, yo) without changing the value of the integral. 
Now we verify that for a circle the integral for the winding number is either +1 or —1, 
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depending on whether the circle is positively or negatively oriented. For a positively 
oriented circle we may use the representation in Equation (11.26). In this case we have 


X(t)=acost+ Xp, Y(t) =asint + yo, 


and the integrand in (11.27) is identically equal to 1. Therefore we obtain 


1 27 
W(a; Po -2[ ldt=1. 


TT J0 


By a similar argument we find that the intergal is —1 when C is negatively oriented. This 
proves that the winding number is either +1 or —1 for a simple closed curve enclosing 
the point (Xo, Vo). 


%11.25 Exercises 


1. Let S = {(x, y) | x? + y? > 0}, and let 


P(x, y) = O(x, y) = 


y —Xx 
x2 4 y2? x? + 2 
if (x, y)€S. Let C be a piecewise smooth Jordan curve lying in S. 
(a) If (0, 0) is inside C, show that the line integral J~ Pdx + Q dy has the value +27, and 
explain when the plus sign occurs. 
(b) Compute the value of the line integral {¢ Pdx + Qdy when (0, 0) is outside C. 
2. Ifr = xi + yyandr = |lr||, let 


ae” n, a(log r) , 
ae aS eer 


Ox 


f(x,y) = 


forr >0. Let C be a piecewise smooth Jordan curve lying in the annulus 1 < x? + y® < 25, 
and find all possible values of the line-integral of f along C. 

3. A connected plane region with exactly one “hole’’ is called doubly connected. (The annulus 
1 <x? + y’ < 2 is an example.) If P and Q are continuously differentiable on an open doubly 
connected region R, and if @P/@y = @Q/ax everywhere in R, how many distinct values are 
possible for line integrals [¢ P dx + Q dy taken around piecewise smooth Jordan curves in R? 

4. Solve Exercise 3 for triply connected regions, that is, for connected plane regions with exactly 
two holes. 

5. Let P and Q be two scalar fields which have continuous derivatives satisfying 0P/dy = 0Q/ 0x 
everywhere in the plane except at three points. Let C,, Cz, C3 be three nonintersecting circles 


having centers at these three points, as shown in Figure 11.21, and let J, = $., Pdx +Qdy. 
Assume that J, = 12, , = 10, /, = 15. 

(a) Find the value of Jo Pdx + Qdy, where C is the figure-eight curve shown. 

(b) Draw another closed curve I’ along which J Pdx + Qdy =1. Indicate on your drawing 
the direction in which I is traversed. 

(c) If 7, = 12,7, =9, and /, = 15, show that there is no closed curve I along which J Pdx + 
Qdy =1. 
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FiGureE 11.21 Exercise 5S. FiGurE 11.22 Exercise 6. 


6. Let, = $, Pdx + Qdy, where 
k 


1 1 1 
PO) Sy Se 
Oy) y= —P+y ety G+)? =| 
and 
x-1 x x+1 

CO = Grit tery Tas Ty 
In Figure 11.22, C, is the smallest circle, x? + y? = $ (traced counterclockwise), C, is the largest 
circle, x? + y® = 4 (traced counterclockwise), and C3 is the curve made up of the three inter- 
mediate circles (x — 1)? + y? =}, x? + y® =}, and (x + 1)? + y® = traced out as shown. 
If J, = 67 and J, = 27, find the value of J,. 


11.26 Change of variables in a double integral 


In one-dimensional integration theory the method of substitution often enables us to 
evaluate complicated integrals by transforming them into simpler ones or into types that 
can be more easily recognized. The method is based on the formula 


(11.29) [? 700 ax = | Fle@le'@ at, 


where a = g(c) and 6 = g(d). We proved this formula (in Volume I) under the assumptions 
that g has a continuous derivative on an interval [c, d] and that fis continuous on the set 
of values taken by g(t) as ¢ runs through the interval [c, d]. 
There is a two-dimensional analogue of (11.29) called the formula for making a change of 
variables in a double integral. It transforms an integral of the form Jf f(x, y) dx dy, 
Ss 


extended over a region S in the xy-plane, into another double integral [{ F(u, v) du dv, 
T 


extended over a new region 7 in the uv-plane. The exact relationship between the regions 
S and 7 and the integrands f(x, y) and F(u, v) will be discussed presently. The method of 
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FiGureE 11.23 The mapping defined by the vector equation r(u, v) = 
X(u, v)i + Yu, v)j. 


substitution for double integrals is more elaborate than in the one-dimensional case 
because there are two formal substitutions to be made, one for x and another for y. This 
means that instead of the one function g which appears in Equation (11.29), we now have 
two functions, say X and Y, which connect x, y with u, v as follows: 


(11.30) x=X(u,v), y= Y(u,v). 


The two equations in (11.30) define a mapping which carries a point (u, v) in the uv-plane 
into an image point (x, y) in the xy-plane. A set T of points in the uwv-plane is mapped onto 
another set S in the xy-plane, as suggested by Figure 11.23. The mapping can also be 
described by means of a vector-valued function. From the origin in the xy-plane we draw 
the radius vector r to a general point (x, y) of S, as shown in Figure 11.23. The vector r 
depends on both wu and v and can be considered a vector-valued function of two variables 
defined by the equation 


(11.31) r(u, v) = X(u, vit Yu, vj if (u,v)EeT. 


This equation is called a vector equation of the mapping. As (u,v) runs through the 
points of 7 the endpoint of r(u, v) traces out the points of S. 

Sometimes the two equations in (11.30) can be solved for u and v in terms of x and y. 
When this is possible we may express the result in the form 


u= U(x,y), v = V(x, y). 
These equations define a mapping from the xy-plane to the wv-plane, called the inverse 


mapping of the one defined by (11.30), since it carries points of S back to T. The so-called 
one-to-one mappings are of special importance. These carry distinct points of T onto distinct 
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points of S; in other words, no two distinct points of T are mapped onto the same point 
of S by a one-to-one mapping. Each such mapping establishes a one-to-one correspondence 
between the points in 7 and those in S and enables us (at least in theory) to go back from S 
to T by the inverse mapping (which, of course, is also one-to-one). 

We shall consider mappings for which the functions X and Y are continuous and have 
continuous partial derivatives 0X/du, 0X/dv, OY/du, and dY/dv on S. Similar assumptions 
are made for the functions U and V. These are not serious restrictions since they are 
satisfied by most functions encountered in practice. 

The formula for transforming double integrals may be written as 


(11.32) [| #0, ») dx dy = [f S1X@, »), Yu, 1 [I, »)| du do. 
S T 


The factor J(u, v) which appears in the integrand on the right plays the role of the factor 
g’(t) which appears in the one-dimensional Formula (11.29). This factor is called the 
Jacobian determinant of the mapping defined by (11.30); it is equal to 


ax ay 
Ou ou 


ax ay] 
Ov Ov 


J(u, v) = 


Sometimes the symbol 0(X, Y)/0(u, v) is used instead of J(u, v) to represent the Jacobian 
determinant. 

We shall not discuss the most general conditions under which the transformation 
formula (11.32) is valid. It can be shown f that (11.32) holds if, in addition to the continuity 
assumptions on X, Y, U, and V mentioned above, we assume that the mapping from T 
to S is one-to-one and that the Jacobian J(u, v) is never zero. The formula is also valid 
if the mapping fails to be one-to-one on a subset of 7 of content zero, or if the Jacobian 
vanishes on a subset of content zero. 

In Section 11.30 we show how the transformation formula (11.32) can be derived as a 
consequence of one of its special cases, namely, the case in which S is a rectangle and the 
function f has the constant value 1 at each point of S. In this special case (11.32) becomes 


(11.33) [{ dx ay = | lsu, v)| du dv. 
S T 


Even for this case a proof is not simple. In Section 11.29 a proof of (11.33) is given with the 
aid of Green’s theorem. The remainder of this section will present a simple geometric 
argument which explains why a formula like (11.33) should hold. 

Geometric motivation for Equation (11.33): Take a region 7 in the uv-plane, as shown in 
Figure 11.23, and let S denote the set of points in the xy-plane onto which 7 is mapped 
by the vector function r given by (11.31). Now introduce two new vector-valued functions 
V, and V, which are obtained by taking the partial derivatives of the components of r 


t See Theorem 10-30 of the author’s Mathematical Analysis. 


Change cf variables in a double integral 395 


v y 


v = constant 
ae aes x = X(u,v) 


u-Curve 


Ficure 11.24 A u-curve and a corresponding velocity vector. 


with respect to u and v, respectively. That is, define 


These vectors may be interpreted geometrically as follows: Consider a horizontal line 
segment in the uv-plane (v is constant on such a segment). The vector function r maps this 
segment onto a curve (called a w-curve) in the xy-plane, as suggested in Figure 11.24. If we 
think of uw as a parameter representing time, the vector V, represents the velocity of the 
position r and is therefore tangent to the curve traced out by the tip of r. In the same way, 
each vector V, represents the velocity vector of a v-curve obtained by setting u = constant. 
A u-curve and a v-curve pass through each point of the region S. 

Consider now a small rectangle with dimensions Au and Av, as shown in Figure 11.25. 
If Au is the length of a small time interval, then in time Au a point of a u-curve moves 
along the curve a distance approximately equal to the product ||V,|| Aw (since ||V,|| repre- 
sents the speed and Aw the time). Similarly, in time Av a point on a v-curve moves a 


VD y 


FiGure 11.25 The image of a rectangular region in the uv-plane is a 
curvilinear parallelogram in the xy-plane. 


396 Multiple integrals 


distance nearly equal to ||V,|| Av. Hence the rectangular region with dimensions Au and 
Av in the uv-plane is traced onto a portion of the xy-plane that is nearly a parallelogram, 
whose sides are the vectors V, Au and V, Av, as suggested by Figure 11.25. The area of 
this parallelogram is the magnitude of the cross product of the two vectors V, Au and 
V, Av; this is equal to 


|\(V, Au) x (Vz Av)|| = ||V, x VI] Au Av. 


If we compute the cross product V, x V, in terms of the components of V, and V, we find 


ij ik 
OX OY 
oe GY a) ea On 
V.xXV,=|o0u du = a ay k = J(u, v)k. 
OX OY o| |a a 
Ov Ov 


Therefore the magnitude of V, x V, is exactly |/(u, v)| and the area of the curvilinear 
parallelogram in Figure 11.25 is nearly equal to |/(u, v)| Au Av. 

If J(u, v) = 1 for all points in 7, then the “‘parallelogram’”’ has the same area as the 
rectangle and the mapping preserves areas. Otherwise, to obtain the area of the parallelo- 
gram we must multiply the area of the rectangle by |/(u, v)|. This suggests that the Jacobian 
may be thought of as a “‘magnification factor” for areas. 

Now let P be a partition of a large rectangle R enclosing the entire region 7 and consider 
a typical subrectangle of P of, say, dimensions Au and Av. If Au and Av are small, the 
Jacobian function J is nearly constant on the subrectangle and hence J acts somewhat like a 
step function on R. (We define J to be zero outside 7.) If we think of J as an actual step 
function, then the double integral of |/| over R (and hence over 7) is a sum of products 
of the form |J(u, v)| Au Av and the above remarks suggest that this sum is nearly equal to 
the area of S, which we know to be the double integral {| dx dy. 

Ss 


This geometric discussion, which merely suggests why we might expect an equation like 
(11.33) to hold, can be made the basis of a rigorous proof, but the details are lengthy and 
rather intricate. As mentioned above, a proof of (11.33), based on an entirely different 
approach, will be given in a later section. 

If J(u, v) = 0 at a particular point (u, v), the two vectors V, and V, are parallel (since 
their cross product is the zero vector) and the parallelogram degenerates into a line segment. 
Such points are called singular points of the mapping. As we have already mentioned, 
transformation formula (11.32) is also valid whenever there are only a finite number of such 
singular points or, more generally, when the singular points form a set of content zero. 
This is the case for all the mappings we shall use. In the next section we illustrate the use of 
formula (11.32) in two important examples. 


11.27 Special cases of the transformation formula 


EXAMPLE 1. Polar coordinates. In this case we write r and @ instead of u and v and 
describe the mapping by the two equations: 


x=rcos#, y=rsiné@. 
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That is, X(r, 6) =rcos@ and Y(r,@)=rsin@. To obtain a one-to-one mapping we 
keep r > 0 and restrict 6 to lie in an interval of the form 6) < 6 < 4) + 27. For example, 
the mapping Is one-to-one on any subset of the rectangle (0, a] x [0, 277) in the r0-plane. 
The Jacobian determinant of this mapping is 


ax ay 
Or or cos 6 sin 6 
J(r, 6) = = = r(cos* 6 4+ sin? 6) =r. 
OX oY —rsin@ rcos@ 
06 «608 


Hence the transformation formula in (11.32) becomes 
ze: y)dx dy = [| £0 cos 6,rsin 6)rdr dé. 
Ss T 


The r-curves are straight lines through the origin and the 9-curves are circles centered at the 
origin. The image of a rectangle in the ré-plane is a “parallelogram’”’ in the xy-plane 


8 = constant 
/ r-curve (8 = constant) 


x =rcos 6 


y=rsin#@ 


— r = Constant 


r 


FicureE 11.26 Transformation by polar coordinates. 


bounded by two radial lines and two circular arcs, as shown in Figure 11.26. The Jacobian 
determinant vanishes when r = 0, but this does not affect the validity of the transformation 
formula because the set of points with r = 0 has content zero. 

Since V, = cos i+ sin 6j, we have ||V,|| = 1, so there 1s no distortion of distances along 
the r-curves. On the other hand, we have 


V, = —rsin6i+rcos 6j, Ae a 


so distances along the 6-curves are multiplied by the factor r. 
Polar coordinates are particularly suitable when the region of integration has boundaries 
along which r or 8 is constant. For example, consider the integral for the volume of one 


398 Multiple integrals 


octant of a sphere of radius a, 


[| Va? =x? — y? ax dy, 


S 


where the region S is the first quadrant of the circular disk x? + y? < a®. In polar co- 
ordinates the integral becomes 


[| va = r?r dr dé, 
T 


where the region of integration T is now a rectangle [0, a] x [0, 37]. Integrating first with 
respect to 6 and then with respect to r we obtain 


Pennetta a _— 2 294 |, ; 
[J Ve =P rar ao = f a eee _ 7a . 
T 2 J0 9) = ; “6 


The same result can be obtained by integrating in rectangular coordinates but the calculation 
is more complicated. 


EXAMPLE 2. Linear transformations. Consider a linear transformation defined by a pair of 
equations of the form 


(11.34) x=Au+ Bv, y=Cu+ Dv, 
where A, B, C, D are given constants. The Jacobian determinant is 
J(u, v) = AD — BC, 


and in order to have an inverse we assume that AD — BC #0). This assures us that the 
two linear equations in (11.34) can be solved for u and v in terms of x and y. 

Linear transformations carry parallel lines into parallel lines. Therefore the image of 
a rectangle in the wv-plane is a parallelogram in the xy-plane, and its area is that of the 
rectangle multiplied by the factor |J(u, v)| = |AD — BC|. Transformation formula (11.32) 
becomes 


[| #0, ») dx dy = 14D — BC| | | f(Au + Bo, Cu + Dv) du do. 
S T 


To illustrate an example in which a linear change of variables is useful, let us consider 
the integral 


) | er OIW+2) dye dy. 
Ss 


where S is the triangle bounded by the line x + y = 2 and the two coordinate axes. (See 
Figure 11.27.) The presence of y — x and y + x in the integrand suggests the change of 
variables 


i—VoX, veytx. 
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FiGureE 11.27 Mapping by a linear transformation. 


Solving for x and y we find 


v—UuU v+u 
x= and y= 
2 2 
The Jacobian determinant is J(u, v) = —%. To find the image T of S in the uwv-plane we 
note that the lines x = 0 and y = 0 map onto the lines u = v and u = —», respectively; 


the line x + y = 2 maps onto the line v = 2. Points inside S satisfy 0 < x + y < 2 and 
these are carried into points of 7 satisfying 0<v <2. Therefore the new region of 
integration 7 is a triangular region, as shown in Figure 11.27. The double integral in 


question becomes 
{| et —/(U+2) Ty dy = - {| e/” du dv. 


S T 


Integrating first with respect to u we find 
2 v 2 
Lf ew du dv = | | edu | dv = -| o(¢ — -) dv =e = 
2 A 2/0 LJ-v 2 Jo e e 


11.28 Exercises 


In each of Exercises 1 through 5, make a sketch of the region S and express the double integral 
SJ f(x, y) dx dy as an iterated integral in polar coordinates. 
S 


= {(x, y)| x? +y? <a}, where a>O. 

= {(x,y) |x? +? < 2x}. 
={(x,y)|a <x? +y? <b}, where 0<a<b. 
={(x,y/O<y<1—-x,0<x<l}. 
={(x,y) |x <y <1, -l <x <]}. 


Ak WN 
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In each of Exercises 6 through 9, transform the integral to polar coordinates and compute its 
value. (The letter a denotes a positive constant.) 


6. Pe + y?) dy | dx. 8. I; | {®. (x? + y*)-4 dy | dx. 


7. [| [® 2 + dy] ae. 9. seo (x + y?) dx | dy. 


In Exercises 10 through 13, transform each of the given integrals to one or more iterated integrals 
in polar coordinates. 


10. [Pf pox» ay] ax. 2 PEP pa, yay] ax. 
11. PUL re + y*)dy| de. 13. aie dy] dx. 


14. Use a suitable linear transformation to evaluate the double integral 
{| (x — y)® sin? (x + y) dx dy 
S 


where S is the parallelogram with vertices (7, 0), (27, 7), (7, 27), (0, 7). 

15. A parallelogram S in the xy-plane has vertices (0, 0), (2, 10), (3, 17), and (1, 7). 
(a) Find a linear transformation u = ax + by, v = cx + dy, which maps S onto a rectangle 
R in the uv-plane with opposite vertices (0, 0) and (4, 2). The vertex (2, 10) should map onto 
a point on the u-axis. 
(b) Calculate the double integral {{ xy dx dy by transforming it into an equivalent integral 
over the rectangle R of part (a). *% 

16. Ifr > 0, let Ir) = ft,e% du. 
(a) Show that /?(r) = ff e—(x*+v") dx dy, where R is the square R = [—r,r] x [—r,r]. 


R 
(b) If C, and C, are the circular disks inscribing and circumscribing R, show that 
{| e@"t4") dx dy < P(r) < {| e(e"+9") dy dy. 
Ci Ce 


(c) Express the integrals over C, and C, in polar coordinates and use (b) to deduce that 
I(r) > Ja as r — o. This proves that [2° e”’ du = J 7/2. 
(d) Use part (c) to deduce that (3) = i a, where I is the gamma function. 

17. Consider the mapping defined by the equations 


x=utv, y=ov-H, 


(a) Compute the Jacobian determinant J(u, v). 
(b) A triangle T in the uv-plane has vertices (0, 0), (2,0), (0,2). Describe, by means of a 
sketch, its image S in the xy-plane. 
(c) Calculate the area of S by a double integral extended over S and also by a double integral 
extended over 7. 
(d) Evaluate ff (x — y + 1)? dx dy. 

S 


18. Consider the mapping defined by the two equations x = u* — v*, y = 2uv. 
(a) Compute the Jacobian determinant J(u, v). 
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(b) Let T denote the rectangle in the uv-plane with vertices (1, 1), (2, 1), (2, 3), (1, 3). De- 
scribe, by means of a sketch, the image S in the xy-plane. 
(c) Evaluate the double integral Jf xy dx dy by making the change of variables x = u? — v’, 


C 
y = 2uv, where C = {(x, y)| x? + y? < 1}. 
19. Evaluate the double integral 
ii J dx — dxdy 
P> r (p? Ee (p2 + x2 + y2)t + y?)? 


over the circular disk R = {(x, y) | x? + y? <r}. Determine those values of p for which 
I(p, r) tends to a limit asr > +0. 


In Exercises 20 through 22, establish the given equations by introducing a suitable change of 
variables in each case. 


20. [| fe + y)dx dy = is flu) du, where S = {(x, y) | |xl +lyl < 1}. 
Ss 
1 rn eee as 
pale [| fax + by +c)dxdy = 2[", J1 — u2 f(ur/a? + b*% +c) du, 
iS 
where S = {(x, y)| x? +y? <1} and a+? #0. 


22. {] f(xy) dx dy = log 2 ie f() du, — where S is the region in the first quadrant bounded by 
Ss 


the curvesxy =1,xy=2,y=x,y =4x. 


11.29 Proof of the transformation formula in a special case 


As mentioned earlier, the transformation formula 
(11.35) |] #0, y) dx dy = [J s1X@, 0), Y@, o)] Jw, 0)] du do 
S T 


can be deduced as a consequence of the special case in which S is a rectangle and f is 
identically 1. In this case the formula simplifies to 


(11.36) [| dx dy =| lJ, v)| du do, 
R R* 


Here R denotes a rectangle in the xy-plane and R* denotes its image in the uv-plane (see 
Figure 11.28) under a one-to-one mapping 


u=U(x,y), v=V(x,y). 
The inverse mapping is given by 


x = X(u, v), y= YU, v), 
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FiGure 11.28 The transformation law for double integrals derived from Green’s theorem. 


and J(u, v) denotes the Jacobian determinant, 


ax ax 
Ou Ov 

J(u, v) = ay Ay 
du av 


In this section we use Green’s theorem to prove (11.36), and in the next section we deduce 
the more general formula (11.35) from the special case in (11.36). 

For the proof we assume that the functions X and Y have continuous second-order partial 
derivatives and that the Jacobian is never 0 in R*. Then J(u, v) is either positive everywhere 
or negative everywhere. The significance of the sign of J(u, v) is that when a point (x, y) 
traces out the boundary of R in the counterclockwise direction, the image point (u, v) 
traces out the boundary of R* in the counterclockwise direction if J(u, v) is positive and in 
the opposite direction if J(u, v) is negative. In the proof we shall assume that J(u, v) > 0. 

The idea of the proof is to express each double integral in (11.36) as a line integral, using 
Green’s theorem. Then we verify equality of the two line integrals by expressing each in 
parametric form. 

We begin with the double integral in the xy-plane, writing 


where Q(x, y) = x and P(x, y) = 0. By Green’s theorem this double integral is equal to 
the line integral 


[ Pax +Qdy =| xdy. 


Here C is the boundary of R traversed in a counterclockwise direction. 
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Similarly, we transform the double integral in the wv-plane into a line integral around the 
boundary C* of R*. The integrand, J(u, v), can be written as follows: 


OXOY OXOY OxXdY x OY  , OY  axdoyY 


Ou dv =v Gus ue: Ou Ov Ovdu dv du 
0 aa x oe 0 “a x ae 
~ aul” ap Ov\ du 
Applying Green’s theorem to the double integral over R* we find 


[J 0) du do= (x = au +x a). 
a c*¥\ a 0 


Therefore, to complete the proof of (11.36) we need only verify that 


(11.37) | x dy - | (x5 au + X 5): 
C o*\ Ou 0 


We introduce a parametrization of C* and use this to find a representation of C. Suppose 
C* is described by a function «@ defined on an interval [a, 5], say 


a(t) = U(t)hi+ V(t)j. 
Let 
Bit) = X[U(t), VOW + YU), VOW. 


Then as ¢ varies over the interval [a, 5], the vector a(t) traces out the curve C* and B(ft) 
traces out C. By the chain rule, the derivative of 8 is given by 


8'(t) = Suot+S voli+ uo + vo. 


Hence 


|. xdy = [, X[U(0, vole u(y 4 a V0) at. 


The last integral over [a, 5] is also obtained by parametrizing the line integral over C* in 
(11.37). Therefore the two line integrals in (11.37) are equal, which proves (11.36). 


11.30 Proof of the transformation formula in the general case 


In this section we deduce the general transformation formula 


(11.38) [] te, ») dx dy = ff 1X, 0), Yu v1 Iu, 0) du do 
S T 
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from the special case treated in the foregoing section, 


(11.39) [{ dx dy =[{ \a(u, »)| du do, 


R R* 


where R is a rectangle and R* is its image in the uv-plane. 
First we prove that we have 


(11.40) [| scx, ») dx dy =| six, 0), Yu, oJ, v)] du do, 
R R* 


where s is any step function defined on R. For this purpose, let P be a partition of R into 
mn subrectangles R,; of dimensions Ax; and Ay,, and let c;; be the constant value that s 
takes on the open subrectangle R,;;. Applying (11.39) to the rectangle R;; we find 


Ax, Ay, =|} dxdy ={{ \s@,0)| du av. 


Ri; Ri;* 


Multiplying both sides by c;; and summing on / and j we obtain 


(11.41) Y SejAxdy, => Sey [f ye, ol du do. 
i=1 j=l rm 


Ri5* 


Since s is a step function, this is the same as 


(11.42) > 


m 
i=l j= 


c,, Ax,;Ay; => > {| s[X(u, v), Y(u, v)] |J(u, v)| du dv. 
i=13=] Pie 


1 


~~ 


Using the additive property of double integrals we see that (11.42) is the same as (11.40). 
Thus, (11.40) is a consequence of (11.39). 

Next we show that the step function s in (11.40) can be replaced by any function / for 
which both sides of (11.40) exist. Let f be integrable over a rectangle R and choose step 
functions s and ¢ satisfying the inequalities 


(11.43) S(x,y) SS(x,¥) S tx, y), 

for all points (x, y) in R. Then we also have 

(11.44) s[X(u, v), Y(u, v)] < f[X(u, v), Y(u, v)] < t[X(u, v), Y(u, v)] 

for every point (u, v) in the image R*. For brevity, write S(u, v) for s[X(u, v), Y(u, v)] and 


define F(u, v) and T(u, v) similarly. Multiplying the inequalities in (11.44) by |J(u, v)| 
and integrating over R* we obtain 


[| stu, 0) lu@, v)] du dv < [J Fu, 0) Ju, v)| du dv < {{ Tu, v) |S, v)| du do. 
R* R* R* 
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Because of (11.40), the foregoing inequalities are the same as 


{| s(x, y}dx dy < {| F(u, v) |J(u, v)| du dv < {| t(x, y) dx dy. 
R R* R 


Therefore ff F(u, v) |J(u, v)| dudv is a number which lies between the integrals 
R* 
Sf s(x, y) dx dy and Sf t(x, y) dx dy for all choices of step functions s and ¢ satisfying (11.43). 
R R 
Since f is integrable, this implies that 


Jl se, y)dx dy = {| F(u, v) |J(u, v)| du dv 
R R* 


and hence (11.38) is valid for integrable functions defined over rectangles. 

Once we know that (11.38) is valid for rectangles we can easily extend it to more general 
regions S by the usual procedure of enclosing S in a rectangle R and extending the function 
f to a new function f which agrees with fon S and has the value 0 outside S. Then we note 
that 


[lr = [{f= [| fixe, v), Y(u, v)] |J(u, v)| du dv = a F(u, v) |J(u, v)| du dv 
iS R R* T 
and this proves that (11.38) is, indeed, a consequence of (11.39). 


11.31 Extensions to higher dimensions 


The concept of multiple integral can be extended from 2-space to n-space for any n > 3. 
Since the development is entirely analogous to the case n = 2 we merely sketch the principal 
results. 

The integrand is a scalar field f defined and bounded on a set S in n-space. The integral 
of f over S, called an n-fold integral, is denoted by the symbols 


ones or [oe [fOas esx) dey s+ dy, 
s Ss 


with n integral signs, or more simply with one integral sign, fg f(x) dx, where x = 
(X1,..++,X,). When n = 3 we write (x, y, z) instead of (x,, x2, x3) and denote triple 


integrals by 
Ne. or [| #0, y, 2) ax dy dz. 
s S 


First we define the n-fold integral for a step function defined on an n-dimensional interval. 
We recall that an n-dimensional closed interval [a, b] is the Cartesian product of n closed 
one-dimensional intervals [a,,5,], where a = (a,,...,a,) and b= (b,,...,5,). An 
n-dimensional open interval (a, b) is the Cartesian product of n open intervals (a,,, b,.). 
The volume of [a, 5] or of (a, b) is defined to be the product of the lengths of the component 
intervals, 


(6, — a)°** (by, — a,). 
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If P,,...,P, are partitions of [a,, 5,],..., [a,, 5,], respectively, the Cartesian product 
P=P,xX°:-: X P, iscalled a partition of [a,b]. A function f defined on [a, b]is called a step 
function if it is constant on each of the open subintervals determined by some partition P. 
The n-fold integral of such a step function 1s defined by the formula 


Jo Jf= ew, 


[a,b] 


where c; is the constant value that f takes on the ith open subinterval and v; is the volume of 
the ith subinterval. The sum is a finite sum extended over all the subintervals of P. 
Having defined the n-fold integral for step functions, we define the integral for more 
general bounded functions defined on intervals, following the usual procedure. Let s and ¢ 
denote step functions such thats < f < ton [a, b]. If there is one and only one number J 


such that 
an Peres fee 


[a,b] [4,5] 


for all choices of s and ¢ satisfying s < f < t, then fis said to be integrable on [a, b], and 
the number / is called the n-fold integral of /, 


haters 


[a,b] 


As in the two-dimensional case, the integral exists if fis continuous on [a, 5]. It also 
exists if fis bounded on [a, b] and if the set of discontinuities of fhas n-dimensional content 
0. A bounded set S has n-dimensional content 0 if for every « > 0 there is a finite collection 
of n-dimensional intervals whose union includes S and the sum of whose volumes does not 
exceed e. 

To define the n-fold integral of a bounded function f over a more general bounded set S, 
we extend f to a new function f which agrees with f on S and has the value zero outside S; 
the integral of f over S is defined to be the integral of f over some interval containing S. 

Some multiple integrals can be calculated by using iterated integrals of lower dimension. 
For example, suppose S is a set in 3-space described as follows: 


(11.45) S={(x,y,z)|(x,y)€Q and g(x,y) <z< galx, y)}, 


where Q is a two-dimensional region, called the projection of S on the xy-plane, and 9, 
¢, are continuous on S. (An example is shown in Figure 11.29.) Sets of this type are bounded 
by two surfaces with Cartesian equations z = (x, y), and z = (x, y) and (perhaps) a 
portion of the cylinder generated by a line moving parallel to the z-axis along the boundary 
of Q. Lines parallel to the z-axis intersect this set in line segments joining the lower surface 
to the upper one. If fis continuous on the interior of S we have the iteration formula 


oylx.y) 


(11.46) [{f res, y, 2) ax dy dz = |] [ [°° poy, 2 dz] ax ay. 
S Q 
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FiGure 11.29 A solid S and its projection Q in the xy-plane. 


That is, for fixed x and y, the first integration is performed with respect to z from the lower 
boundary surface to the upper one. This reduces the calculation to a double integral over the 
projection Q, which can be treated by the methods discussed earlier. 

There are two more types of sets analogous to those described by (11.45) in which the x- 
and y-axes play the role of the z-axis, with the projections taken in the yz- or xz-planes, 
respectively. Triple integrals over such sets can be computed by iteration, with formulas 
analogous to (11.46). Most 3-dimensional sets that we shall encounter are either of one of 
the three types just mentioned or they can be split into a finite number of pieces, each of 
which is of one of these types. 

Many iteration formulas exist for n-fold integrals when n > 3. For example, if Q is a 
k-dimensional interval and R an m-dimensional interval, then an (m + k)-fold integral over 
Q xX Ris the iteration of an m-fold integral with a k-fold integral, 


Core eoico oo ep ee 
QxR Q R 


provided all the multiple integrals in question exist. Later in this chapter we illustrate the 
use of iterated multiple integrals in computing the volume of an n-dimensional sphere. 


11.32 Change of variables in an n-fold integral 


The formula for making a change of variables in a double integral has a direct extension 
for n-fold integrals. We introduce new variables u,,..., u, related to x,,..., x, by n 
equations of the form 


X, = X1(u,,...,U,), * 9 Xn = gllags oat tl) 
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Let x = (X1,...,%,), 4 = (uy,...,u,), and X = (X,,..., X,). Then these equations 
define a vector-valued mapping 


xX:T—S 
from a set Tin n-space to another set Sin n-space. We assume the mapping X is one-to-one 


and continuously differentiable on JT. The transformation formula for n-fold integrals 
assumes the form 


(11.47) [ fx) dx = | f{X(w)] [det DX(u)| du, 


where DX(u) = [D,X,,(u)] is the Jacobian matrix of the vector field X. In terms of com- 
ponents we have 
D,X,(u) D,X,(u) +++ D,X,(u) 


DX(u) = 
D,X,,(u) D,X,(u) poate D,X,(u) 


As in the two-dimensional case, the transformation formula is valid if X is one-to-one on 
T and if the Jacobian determinant J(u) = det DX(u) is never zero on T. It is also valid if 
the mapping fails to be one-to-one on a subset of T having n-dimensional content zero, or if 
the Jacobian determinant vanishes on such a subset. 

For the three-dimensional case we write (x, y, z) for (x1, X2, X3), (u, v, w) for (uy, Uy, Us), 
and (X, Y, Z) for (X,, X,, X3). The transformation formula for triple integrals takes the 
form 


(11.48) | {| f(x,y, 2) dx dy dz 
S 


= {ff rxw v, Ww), Y(u, v, w), Z(u, v, w)] |J(u, v, w)| du dv dw, 
T 


where J(u, v, w) is the Jacobian determinant, 

ax OY az 
Ou Ou Ou 
ax aY a 
Ov Ov Ovi. 
ax aY a 
Ow Ow Ow 


J(u, v, w) = 


In 3-space the Jacobian determinant can be thought of as a magnification factor for volumes. 
In fact, if we introduce the vector-valued function r defined by the equation 


r(u,v,w) = X(u,v, wit Y(u, v, w)j + Z(u, v, wk, 
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and the vectors 


Or Ox oY OZ 
y= a S54 G4 Sk, 
‘Ou oe ane! on 


Or OX. OY... @Z 
eae 
Ww Ww Ow Ow 


an argument similar to that given in Section 11.26 suggests that a rectangular parallelepiped 
of dimensions Au, Av, Aw in wvw-space is carried onto a solid which is nearly a curvilinear 
“parallelepiped”’ in xyz-space determined by the three vectors V, Au, V, Av, and V; Aw. 
(See Figure 11.30.) The boundaries of this solid are surfaces obtained by setting u = 
constant, v = constant, and w = constant, respectively. The volume of a parallelepiped 
is equal to the absolute value of the scalar triple product of the three vectors which determine 
it, so the volume of the curvilinear parallelepiped is nearly equal to 


|((V, Au): (V, Av) x (V3 Aw)| = |V,° V, xX V3| Au Av Aw = |J(u, v, w)| Au Av Aw. 
11.33. Worked examples 
Two important special cases of (11.48) are discussed in the next two examples. 


EXAMPLE |. Cylindrical coordinates. Here we write r, 0, z for u, v, w and define the 
mapping by the equations 


(11.49) x =rcos6, y=rsing, ae 


Z = constant 


x=rcos8 
y=rsin@ 
Z=2 
—_______—_—_—— i 


8 = constant 


x 


FicurE 11.30 Transformation by cylindrical coordinates. 
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In other words, we replace x and y by their polar coordinates in the xy-plane and leave 
z unchanged. Again, to get a one-to-one mapping we must keep r > 0 and restrict 0 to 
be in an interval of the form 0) < 0 < 6) + 27. Figure 11.30 shows what happens to a 
rectangular parallelepiped in the r@z-space. 

The Jacobian determinant of the mapping in (11.49) is 


cos 0 sinOd 0O 
J(r, 0, Z) =|—rsinO rcos#? 0]/= r(cos? 6 + sin? 0) =r, 
0 0 l 


and therefore the transformation formula in (11.48) becomes 
[re y, z)dx dy dz = [{] re cos 6,rsin 0, z)rdrd6 dz. 
S T 


The Jacobian determinant vanishes when r = 0, but this does not affect the validity of the 
transformation formula because the set of points with r = 0 has 3-dimensional content 0. 


EXAMPLE 2. Spherical coordinates. In this case the symbols p, 6, y are used instead of 
u, v, w and the mapping is defined by the equations 


x=pcos@sing, y=psn@sing, z=pcos®@. 


The geometric meanings of p, 8, and » are shown in Figure 11.31. To get a one-to-one 
mapping we keep p>0,0<6< 27, and0 << y< 7. The surfaces p = constant are 
spheres centered at the origin, the surfaces 6 = constant are planes passing through the 
z-axis, and the surfaces » = constant are circular cones with their axes along the z-axis. 
Therefore a rectangular box in p0g-space is mapped onto a solid of the type shown in 
Figure 11.31. 


Y Zz 


P = constant 
8 = constant 


x = psin Y cos 6 (x,y,2Z) 
y=psin & sin 0 


Z= pcos P 


Y = constant 


“‘ 


we Mg 
p sin P 


Ficure 11.31 Transformation by spherical coordinates. 
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The Jacobian determinant of the mapping is 


cos 0 sin @ sin 0 sin » cos p 
J(p, 9, ~) =|—psinOsing pcos Q@sin » 0 = —p* sin . 
pcos@cosm psinfcosm —psin 


Since sin g > 0 if 0 < w < 7, we have |J/(p, 9, g)| = p*? sin m and the formula for trans- 
forming triple integrals becomes 


[fre y, z)dx dy dz = ai F(p, 9, v)p* sin y dp dO dq, 
Ss T 


where F(p, 0, vy) = f(p cos 6 sin ¢, p sin @ sin y, pcos g). Although the Jacobian deter- 
minant vanishes when g = 0 the transformation formula is still valid because the set of 
points with g = 0 has 3-dimensional content 0. 


The concept of volume can be extended to certain classes of sets (called measurable sets) 
in n-space in such a way that if S is measurable then its volume is equal to the integral of the 
constant function 1 over S. That is, if v(S) denotes the volume of S, we have 


o(S)= |---| dxy--- dx, 
S 


We shall not attempt to describe the class of sets for which this formula is valid. Instead, 
we illustrate how the integral can be calculated in some special cases. 


EXAMPLE 3. Volume of an n-dimensional interval. If S is an n-dimensional interval, say 
S = [a,,b,] x -+: X [a,,5,], the multiple integral for v(S) is the product of n one- 
dimensional integrals, 


by Dn 
oS) = J days - [P dx, = (Bi — a4) +++ (by = ay). 
This agrees with the formula given earlier for the volume of an n-dimensional interval. 


EXAMPLE 4. The volume of an n-dimensional sphere. Let S,,(a) denote the n-dimensional 
solid sphere (or n-ball) of radius a given by 


S,(a) = {(%1,---5%n) | sd °* + x5 < a*}, 
and let 


V(a) = Jo | dys dey, 


the volume of S,,(a). We shall prove that 


72 
(11.50) V,(a) = ——— a 
T(gn + 1) 


n 
? 
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where [is the gamma function. Form = | the formula gives V,(a) = 2a, the length of the 
interval [—a, a]. For n = 2 it gives V,(a) = 7a*, the area of a circular disk of radius a. 
We shall prove (11.50) for n > 3. 

First we prove that for every a > 0 we have 


(11.51) V_(a) = a"V,(1). 


In other words, the volume of a sphere of radius a is a” times the volume of a sphere of 
radius 1. To prove this we use the linear change of variable x = au to map S,(1) onto 
S,,(a). The mapping has Jacobian determinant a”. Hence 


V,(a) = [++ | dxy--+dx, = [++ { a" dus du, = a"V,(1). 
Sn (a) Sn(1) 
This proves (11.51). Therefore, to prove (11.50) it suffices to prove that 
gt? 


(11.52) V(1) = Tae 


First we note that x? +--+ + x? <1 if and only if 


itee tee, ci —ve_,—x and 148 <1. 


Therefore we can write the integral for V,,(1) as the iteration of an (n — 2)-fold integral and 
a double integral, as follows: 


ass) v= ff [ fo] dx, ° ++ dx yg] Dna Ay. 


2 2 2 2 2 
Ln-1t+%nLl Lite ++ 2n-251—Xn-1—Xn 


The inner integral is extended over the (nm — 2)-dimensional sphere S,_,(R), where 
R=V1—x 


n—l 


— x? so it is equal to 
VR) = R”*V,,_o(1) = (1 — xp — xn)? *V of 1). 
Now we write x for x,_, and y for x,. Then (11.53) becomes 
V,(1) = V,_2(1) ia (1 — x? — y?)"/2-1 dx dy. 
ay? <1 
We evaluate the double integral by transforming it to polar coordinates and obtain 
27 1 / Ir 
V,(4) = V(t) { (i —r°)"?"rdrdé = V,_(1)—. 
0 0 n 
In other words, the numbers V,,(1) satisfy the recursion formula 


VS Hea: 
n 
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But the sequence of numbers {f(n)} defined by 


nl? 


1) = Rand 


satisfies the same recursion formula because Dis + 1) =sI(s). Also, (4) = V9 (see 
Exercise 16, Section 11.28), so ['(3) = $V 7 and f(1) = V,(1) = 2. Also, f(2) = V2(1) = 7, 
hence we have f(n) = V,,(1) for all > 1. This proves (11.52). 


11.34 Exercises 


Evaluate each of the triple integrals in Exercises 1 through 5. Make a sketch of the region of 
integration in each case. You may assume the existence of all the integrals encountered. 


1. fff xy’z3 dx dy dz, where S is the solid bounded by the surface z = xy and the planes 
rs 


y=x,x =I1,andz =0. 
2. ffpd +x +y +2) % dx dy dz, where S is the solid bounded by the three coordinate 
Ss 


planes and the planex +y +z =1. 
3. [aye ex dy'de: where S = {(x,y,z)| x? +y? +22? <1,x>0,y>0,z>0}. 


4. Meee Ae og. ht ag _ dx dydz, where S is the solid bounded by the ellipsoid 


5. ff Vx? + y? dx dy dz, where S is the solid formed by the upper nappe of the cone 
Ss 
z? = x2 + y* and the plane z = 1. 
In Exercises 6, 7, and 8, a triple integral [ff f(x, y, z) dx dy dz of a positive function reduces 


to the iterated integral given. In each case describe the region of integration S by means of a 
sketch, showing its projection on the xy-plane. Then express the triple integral as one or more 
iterated integrals in which the first integration is with respect to y. 


6. [fe |" few. y, 2) dz lay) ae. 
ve cl — = es f(x, y, z) dz dy) dx. 


«POLL ron.) at] a 
WL ro] a) =3], & oro 


9. Show that: 
Evaluate the integrals in Exercises 10, 11, and 12 by transforming to cylindrical coordinates. You 
may assume the existence of all integrals encountered. 


10. fff (x? + y?) dx dy dz, where S is the solid bounded by the surface x? + y? = 2z and the 
Ss 


plane z = 2. 
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11. fff dx dy dz, where S is the solid bounded by the three coordinate planes, the surface 
S 


z= x*+ y*, and the planex +y =1. 
12. Sff Gy? + 2?) dx dy dz, where S is a right circular cone of altitude A with its base, of radius 
S 


a, in the xy-plane and its axis along the z-axis. 
Evaluate the integrals in Exercises 13, 14, and 15 by transforming to spherical coordinates. 


13. [Jf dx dy dz, where S is a solid sphere of radius a and center at the origin. 
Ss 


14. JJJ dx dy dz, where S is the solid bounded by two concentric spheres of radii a and 5, 
rs 


where 0 <a < 5, and the center is at the origin. 
15. fff (i — a)? + (Cy — b)? + (2 — 0)? 4 dx dy dz, where S' is a solid sphere of radius R 
Ss 


and center at the origin, and (a, 5, c) is a fixed point outside this sphere. 
16. Generalized spherical coordinates may be defined by the following mapping: 


x =apcos™ Osin" 9g, y = bpsin™ 6 sin” 9, Z =cpcos” g, 
where a, b, c, m, and n are positive constants. Show that the Jacobian determinant is equal to 
—abcmn p2 cos™—! 6 sin™—! 6 cos"! » sin?”“! g, 


Triple integrals can be used to compute volume, mass, center of mass, moment of inertia, and 
other physical concepts associated with solids. If S is a solid, its volume V is given by the triple 
integral 


V= a faa 


If the solid is assigned a density f(x, y, z) at each of its points (x, y, z) (mass per unit volume), its 
mass M is defined to be 


M= [| [rey z) dx dy dz, 
S 


and its center of mass the point (x, ¥, Z) with coordinates 


l 
x= wz ||| xfov. z) dx dy dz, 
Ss 
and so on. The moment of inertia J,,, about the xy-plane is defined by the equation 
Ley = {J 2 f(x, y, z) dx dy dz 
Ss 


and similar formulas are used to define /,, and J,,.. The moment of inertia J; about a line L is 
defined to be 


I, = [ [[erey. z) f(x, y, 2) dx dy dz, 
rs 
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where 6(x, y, z) denotes the perpendicular distance from a general point (x, y, z) of Sto the line L. 


17. 


30. 


31. 


Show that the moments of inertia about the coordinate axes are 


I, = L yy =i Lys lL, — Lyx a foes I, — Loy = ee 


. Find the volume of the solid bounded above by the sphere x? + y? + z? = 5 and below by 


the paraboloid x? + y? = 4z. 


. Find the volume of the solid bounded by the xy-plane, the cylinder x? + y? = 2x, and the 


cone z = ./x? + y?. 


. Compute the mass of the solid lying between two concentric spheres of radii a and b, where 


0 <a <b, if the density at each point is equal to the square of the distance of this point from 
the center. 


. A homogeneous solid right circular cone has altitude A. Prove that the distance of its centroid 


from the base is //4. 


. Determine the center of mass of a right circular cone of altitude h if its density at each point 


is proportional to the distance of this point from the base. 


. Determine the center of mass of a right circular cone of altitude h if its density at each point is 


proportional to the distance of this point from the axis of the cone. 


. A solid is bounded by two concentric hemispheres of radii a and b, where 0 <a < b. If the 


density is constant, find the center of mass. 


. Find the center of mass of a cube of side / if its density at each point is proportional to the 


square of the distance of this point from one corner of the base. (Take the base in the xy-plane 
and place the edges on the coordinate axes.) 


. A right circular cone has altitude /, radius of base a, constant density, and mass M. Find its 


moment of inertia about an axis through the vertex parallel to the base. 


. Find the moment of inertia of a sphere of radius R and mass M about a diameter if the density 


is constant. 


. Find the moment of inertia of a cylinder of radius a and mass M if its density at each point is 


proportional to the distance of this point from the axis of the cylinder. 


. The stem of a mushroom is a right circular cylinder of diameter 1 and length 2, and its cap 


is a hemisphere of radius R. If the mushroom is a homogeneous solid with axial symmetry, 
and if its center of mass lies in the plane where the stem joins the cap, find R. 

A new space satellite has a smooth unbroken skin made up of portions of two circular cylinders 
of equal diameters D whose axes meet at right angles. It is proposed to ship the satellite to 
Cape Kennedy in a cubical packing box of inner dimension D. Prove that one-third of the box 
will be waste space. 

Let S,,(a) denote the following set in n-space, where a > 0: 


Sra) = {(x%15+-- 5 Xn) | lal +°°+ + len] <a}. 


When n = 2 the set is a square with vertices at (0, ta) and (+a,0). When xv = 3 it is an 
octahedron with vertices at (0,0, +a), (0, +a,0),and (+a, 0,0). Let V,,(a) denote the volume 
of S,,(a), given by 


V,(a) = fees | days dig. 
(a) Prove that V,(a) = a”"V,(1). 


(b) For n > 2, express the integral for V,,(1) as an iteration of a one-dimensional integral and 
an (n — 1)-fold integral and show that 


2 
Va(l) = Vacs(l) [FU = bade = 5 VD. 
-1 
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npn 


(c) Use parts (a) and (b) to deduce that V,(a) = 7 


32. Let S,,(a) denote the following set in n-space, where a > 0 andn > 2: 
Sr(a) = {(%1,---5 Xn) [lel +1x,| <a foreachi =1,...,n — 1}. 


(a) Make a sketch of S,(1) when n = 2 and when n = 3. 
(b) Let V,(a) = §---J§dx,--+-+dx,, and show that V,(a) = a"V,(1). 


Sn(a) 
(c) Express the integral for V,,(1) as an iteration of a one-dimensional integral and an (n — 1)- 
fold integral and deduce that V,,(a) = 2"a"/n. 
33. (a) Refer to Example 4, p. 411. Express the integral for V,(1), the volume of the x-dimensional 
unit sphere, as the iteration of an (nm — 1)-fold integral and a one-dimensional integral and 
thereby prove that 


1 
V,(1) = 2V, | (1 — x2)("-D/2 dy, 
0 


(b) Use part (a) and Equation (11.52) to deduce that 


n+1 
ie oe r( 9) ) 
, —_—_—_—_———— . 


Va 
cos” t dt => 
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SURFACE INTEGRALS 


12.1 Parametric representation of a surface 


This chapter discusses surface integrals and their applications. A surface integral can be 
thought of as a two-dimensional analog of a line integral where the region of integration is a 
surface rather than a curve. Before we can discuss surface integrals intelligently, we must 
agree on what we shall mean by a surface. 

Roughly speaking, a surface is the locus of a point moving in space with two degrees 
of freedom. In our study of analytic geometry in Volume I we discussed two methods 
for describing such a locus by mathematical formulas. One is the implicit representation 
in which we describe a surface as a set of points (x, y, Z) satisfying an equation of the 
form F(x, y, Zz) = 0. Sometimes we can solve such an equation for one of the coordinates 
in terms of the other two, say for z in terms of x and y. When this is possible we obtain 
an explicit representation given by one or more equations of the form z = f(x, y). For 
example, a sphere of radius 1 and center at the origin has the implicit representation 
x? + y? + z7—1=0. When this equation is solved for z it leads to two solutions, 
z=V1—x?- y? and z= Aa y®. The first gives an explicit representation 
of the upper hemisphere and the second of the lower hemisphere. 

A third method for describing surfaces is more useful in the study of surface integrals; 
this is the parametric or vector representation in which we have three equations expressing 
x, y, and z in terms of two parameters u and v: 


(12.1) x = X(u, v), y= Yu, v), z= Z(u, v). 


Here the point (u, v) is allowed to vary over some two-dimensional connected set T in 
the uv-plane, and the corresponding points (x, y, Z) trace out a surface in xyz-space. This 
method for describing a surface is analogous to the representation of a space curve by three 
parametric equations involving one parameter. The presence of the two parameters in 
(12.1) makes it possible to transmit two degrees of freedom to the point (x, y, z), as suggested 
by Figure 12.1. Another way of describing the same idea is to say that a surface is the image 
of a plane region T under the mapping defined by (12.1). 

If we introduce the radius vector r from the origin to a general point (x, y, z) of the 
surface, we may combine the three parametric equations in (12.1) into one vector equation 


417 
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FiGure 12.1 Parametric representation of a surface. 


of the form 


(12.2) r(u, v) = X(u, vi + Yu, vj + Zu, v)k, where (u,v)ET. 


This is called a vector equation for the surface. 

There are, of course, many parametric representations for the same surface. One of 
these can always be obtained from an explicit form z = f(x, y) by taking X(u, v) = u, 
Y(u, v) =v, Z(u, v) = f(u, v). On the other hand, if we can solve the first two equations 


in (12.1) for u and v in terms of x and y and substitute in the third—we obtain an explicit 
representation z = f(x, y). 


EXAMPLE |. A parametric representation of a sphere. The three equations 


(12.3) xX = acosucosv, y=asinucosv, Z=asinv 


FIGURE 12.2 Parametric representation of a sphere. 
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FiGurE 12.3 Deformation of a rectangle into a hemisphere. 


serve aS parametric equations for a sphere of radius a and center at the origin. If we 
square and add the three equations in (12.3) we find x? + y? + z? = a?, and we see that 
every point (x, y, z) satisfying (12.3) lies on the sphere. The parameters u and v in this 
example may be interpreted geometrically as the angles shown in Figure 12.2. If we let 
the point (u, v) vary over the rectangle T = [0, 27] x [— $7, 37], the points determined 
by (12.3) trace out the whole sphere. The upper hemisphere is the image of the rectangle 
[0, 27] x [0, 7] and the lower hemisphere is the image of [0, 277] x [—47, 0]. Figure 12.3 
gives a concrete idea of how the rectangle [0, 27] x [0, $7] is mapped onto the upper 
hemisphere. Imagine that the rectangle is made of a flexible plastic material capable of 
being stretched or shrunk. Figure 12.3 shows the rectangle being deformed into a hemi- 
sphere. The base AB eventually becomes the equator, the opposite edges AD and BC are 
brought into coincidence, and the upper edge DC shrinks to a point (the North Pole). 


EXAMPLE 2. A parametric representation of a cone. The vector equation 
r(u, v) = vsinacosui+vsinasinuj +vcosak 


represents the right circular cone shown in Figure 12.4, where « denotes half the vertex 
angle. Again, the parameters u and v may be given geometric interpretations; v is the 
distance from the vertex to the point (x, y, z) on the cone, and uw is the polar-coordinate 
angle. When (u, v) is allowed to vary over a rectangle of the form [0, 27] x [0, A], the 
corresponding points (x, y, z) trace out a cone of altitude hcos«. A plastic rectangle 


x 


FIGURE 12.4 Parametric representation of a FIGURE 12.5 Deformation of a rectangle 
cone. into a cone. 
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may be physically deformed into the cone by bringing the edges AD and BC into coin- 
cidence, as suggested by Figure 12.5, and letting the edge AB shrink to a point (the vertex 
of the cone). The surface in Figure 12.5 shows an intermediate stage of the deformation. 


In the general study of surfaces, the functions X, Y, and Z that occur in the parametric 
equations (12.1) or in the vector equation (12.2) are assumed to be continuous on 7. The 
image of T under the mapping r is called a parametric surface and will be denoted by the 
symbol r(T). In many of the examples we shall discuss, T will be a rectangle, a circular 
disk, or some other simply connected set bounded by a simple closed curve. If the function 
r is one-to-one on T, the image r(7) will be called a simple parametric surface. In such a 
case, distinct points of T map onto distinct points of the surface. In particular, every 
simple closed curve in T maps onto a simple closed curve lying on the surface. 

A parametric surface r(T) may degenerate to a point or to a curve. For example, if all 
three functions X, Y, and Z are constant, the image r(T) is a single point. If X, Y, and Z 
are independent of v, the image r(T) is a curve. Another example of a degenerate surface 
occurs when X(u,v)=u+v, Y(u,v) = (u+v)?, and Z(u,v) = (u+v)?, where 
T = [0,1] x [0,1]. If we write ¢ = u + v we see that the surface degenerates to the space 
curve having parametric equations x = ?t, y= ??, and z= #3, whereO <t <2. Such 
degeneracies can be avoided by placing further restrictions on the mapping function r, 
as described in the next section. 


12.2 The fundamental vector product 


Consider a surface described by the vector equation 
r(u, v) = X(u, v)ji + Y(u, v)j + Zu, v)k, where (u,v) ET. 
If X, Y, and Z are differentiable on 7’ we consider the two vectors 


Or Ox oY OZ 
Or _OX,, OY. , OZ, 
Ou a Oe 


and 

Or Ox. , OY OZ 

—=—i+—Jt—k. 

dv dv av” av 
The cross product of these two vectors Or/du x Or/dv will be referred to as the fundamental 
vector product of the representation r. Its components can be expressed as Jacobian 
determinants. In fact, we have 


i ij ok 
: ay az| az ax| ax ay 
ap Op OX oY dz Ou Ou Ou ou Ou Ou 
(12.4) —x—=|d0u ou du\= i+ jt k 
au” ae ay az|\|az ax|’ lax ay 
OX OY oZ Ov Ov Ov Ov Ov av 
Ov dv Ov 


4% 2), ZX), ALY, 
O(u, v) O(u, v) O(u, v) 
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FIGURE 12.6 Geometric interpretation of the vectors — , — , and — xX —-. 
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If (u, v) is a point in T at which Or/du and Or/dv are continuous and the fundamental 
vector product is nonzero, then the image point r(u, v) is called a regular point of r. 
Points at which dr/du or Or/dv fails to be continuous or dr/du x Or/dv = O are called 
singular points of r. A surface r(T) is called smooth if all its points are regular points. 
Every surface has more than one parametric representation. Some of the examples 
discussed below show that a point of a surface may be a regular point for one repre- 
sentation but a singular point for some other representation. The geometric significance 
of regular and singular points can be explained as follows: 

Consider a horizontal line segment in T. Its image under r is a curve (called a u-curve) 
lying on the surface r(7). For fixed v, think of the parameter u as representing time. The 
vector Or/du is the velocity vector of this curve. When u changes by an amount Au, a point 
originally at r(u, v) moves along a u-curve a distance approximately equal to ||dr/du| Au 
since ||Or/du|| represents the speed along the u-curve. Similarly, for fixed u a point of a 
v-curve moves in time Av a distance nearly equal to ||dr/du| Av. A rectangle in T having 
area Au Av is traced onto a portion of r(T) which we shall approximate by the parallelogram 
determined by the vectors (dr/du) Au and (Or/dv) Av. (See Figure 12.6.) The area of the 
parallelogram spanned by (dr/du) Au and (dr/dv) Av is the magnitude of their cross 
product, 

Or _ or 
ee 
Ou dv 


Au x —Av 


Au Av. 
Ou Ov a 


ay x 3 no = 


Therefore the length of the fundamental vector product may be thought of as a local 
magnification factor for areas. At the points at which this vector product is zero the 
parallelogram collapses to a curve or point, and degeneracies occur. At each regular 
point the vectors Or/du and Or/dv determine a plane having the vector Or/du x Or/dv as 
anormal. In the next section we shall prove that dr/du x Or/dv is normal to every smooth 
curve on the surface; for this reason the plane determined by dr/du and Or/dv is called the 
tangent plane of the surface. Continuity of dr/du and Or/dv implies continuity of dr/du x 
Or/dv; this, in turn, means that the tangent plane varies continuously on a smooth surface. 
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Thus we see that continuity of dr/du and or/dv prevents the occurrence of sharp edges or 
corners on the surface; the nonvanishing of or/du x Or/dv prevents degeneracies. 


EXAMPLE |. Surfaces with an explicit representation, z = f(x, y). For a surface with an 
explicit representation of the form z = f(x, y), we can use x and y as the parameters, 
which gives us the vector equation 


r(x, y) = xi + yi +f Wk. 


This representation always gives a simple parametric surface. The region T is called the 
projection of the surface on the xy-plane. (An example is shown in Figure 12.7, p. 425.) 
To compute the fundamental vector product we note that 


Or ., of or .,, of 
—S — ae k d —_ = poe k, 
eae ee ae, 
if fis differentiable. This gives us 
i j ik 
of 
Or Or 10 — of of 
12.5 — xX — = 0 —~j——jtkh. 
ee) oy : a ae 
012 
oy 


Since the z-component of dr/dx x Or/dy is 1, the fundamental vector product is never zero. 
Therefore the only singular points that can occur for this representation are points at 
which at least one of the partial derivatives Of/0x or df/dy fails to be continuous. 

A specific case is the equation z = Vl —x#- y”, which represents a hemisphere of 
radius | and center at the origin, if x? + y? << 1. The vector equation 


r(x, y) =xit yp + V1 —x2— yk 


maps the unit disk T= {(x, y)|x? + y? <1} onto the hemisphere in a one-to-one 
fashion. The partial derivatives dr/dx and Or/dy exist and are continuous everywhere in 
the interior of this disk, but they do not exist on the boundary. Therefore every point 
on the equator is a singular point of this representation. 


EXAMPLE 2. We consider the same hemisphere as in Example 1, but this time as the image 
of the rectangle T = [0, 27] x [0, $7] under the mapping 


r(u, v) = acosucosvi+asinucosvj+asinuvk. 
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The vectors Or/du and Or/dv are given by the formulas 


or . : 
— = —asinucosvi+acosucos vj, 
Ou 
Or 
Ov 


= —acosu sinvi —asinu sinvj + acosvk. 


An easy calculation shows that their cross product is equal to 


cd x Lee) v) 

Ou dv — 

The image of T is not a simple parametric surface because this mapping is not one-to-one 
on T. In fact, every point on the line segment v = 47, 0 < u < 27, is mapped onto 
the point (0,0, a) (the North Pole). Also, because of the periodicity of the sine and 
cosine, r takes the same values at the points (0, v) and (27, v), so the right and left edges 
of T are mapped onto the same curve, a circular arc joining the North Pole to the point 
(a, 0,0) on the equator. (See Figure 12.3.) The vectors dr/du and Or/dv are continuous 
everywhere in 7. Since ||0r/Ou x Or/dv|| = a® cos v, the only singular points of this repre- 
sentation occur when cos v = 0. The North Pole is the only such point. 


12.3. The fundamental vector product as a normal to the surface 


Consider a smooth parametric surface r(T), and let C* be a smooth curve in T. Then 
the image C = r(C*) is a smooth curve lying on the surface. We shall prove that at each 
point of C the vector dr/du x Or/dv is normal to C, as illustrated in Figure 12.6. 

Suppose that C* is described by a function @ defined on an interval [a, b], say 


a(t) = U(t)i+ V(t). 
Then the image curve C is described by the composite function 
e(t) = rla(t)] = Xle()l + Vial + Zla@)k. 


We wish to prove that the derivative p’(t) is perpendicular to the vector Or/du x or/dv 
when the partial derivatives Or/du and Or/dv are evaluated at (U(t), V(t)). To compute 
e'(t) we differentiate each component of e(t) by the chain rule (Theorem 8.8) to obtain 


(12.6) e'(t) = VX-al(thi+ VY: a'(t)j + VZ-a'(dk, 


where the gradient vectors VX, V Y, and VZ are evaluated at (U(t), V(t)). Equation (12.6) 
can be rewritten as 


ei =2um+ “v0, 
Ou Ov 


where the derivatives 0r/du and Or/dv are evaluated at (U(t), V(t)). Since dr/du and or/dv 
are each perpendicular to the cross product dr/du x or/dv, the same is true of e’(t). This 


424 Surface integrals 


proves that dr/du x Or/dv is normal to C, as asserted. For this reason, the vector product 
Or/Ou X Or/dv is said to be normal to the surface r(T). At each regular point P of r(T) the 
vector Or/Ou x Or/dv is nonzero; the plane through P having this vector as a normal is 
called the tangent plane to the surface at P. 


12.4 Exercises 


In Exercises 1 through 6, eliminate the parameters u and v to obtain a Cartesian equation, thus 
showing that the given vector equation represents a portion of the surface named. Also, compute 
the fundamental vector product or/0u x Or/ dv in terms of u and v. 


1. Plane: 
r(u,v) = (Xo + au + byv)i + (Vo + agu + bov)j + (Zp + agu + bgv)k. 
2. Elliptic paraboloid: 
r(u,v) =aucosvi + businvj + u7k. 
3. Ellipsoid: 
r(u,v) =asinucosvit+ bsinusinvj +ccosuk. 
4. Surface of revolution: 
r(u,v) =ucosvi tusinvj + f(wk. 
5. Cylinder: 
r(u,v) =ui tasinvuj tacosuvk. 
6. Torus: 
r(u,v) = (a+ bcosu)sinvi + (a + bcosu)cosvj + bsin uk, where 0 <b <a. Whatare 
the geometric meanings of a and 5? 


In Exercises 7 through 10 compute the magnitude of the vector product Or/@u x Or/ dv. 


. r(u,v) =asinucoshvi + bcosucoshvj +csinhvk. 
ru, v) = (u F v)i + (u — vj + 4v°k. 

Jru,v) =(utoviit (+07 + (8 + Dk. 

.r(u,v) =ucosvi+usinvyg + 4u* sin 2vk. 


Ow oO ~ 


12.5 Area of a parametric surface 


Let S = r(T) be a parametric surface described by a vector-valued function r defined 
on a region T in the uv-plane. In Section 12.2 we found that the length of the fundamental 
vector product dr/du x Or/dv could be interpreted as a local magnification factor for areas. 
(See Figure 12.6.) A rectangle in T of area Au Av is mapped by r onto a curvilinear paral- 
lelogram on S with area nearly equal to 


This observation suggests the following definition. 


Or or 


a x > Au Av. 


DEFINITION OF AREA OF A PARAMETRIC SURFACE. The area of S, denoted by a(S), is defined 
by the double integral 


(12.7) a(S) = | ne 


Ls du dv. 


Ou Ov 
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In other words, to determine the area of S we first compute the fundamental vector 


product 0r/du x or/dv and then integrate its length over the region 7. When Or/du x Or/dv 
is expressed in terms of its components, by means of Equation (12.4), we have 


2 2 
(12.8) a(S) = I y( a2) 2) + (") + oo) du dv. 
es 7 O(u, v) O(u, v) 
Written in this form, the integral for surface area resembles the integral for computing 
the arc length of a curve.t 


If S is given explicitly by an equation of the form z = f(x, y) we may use x and y as 
the parameters. The fundamental vector product is given by Equation (12.5), so we have 


“|-¥-Besl GG 


In this case the integral for surface area becomes 


(12.9) a(S) = i++ 4 t) + () aiiiy 


where the region T is now the projection of S on the xy-plane, as illustrated in Figure 12.7. 
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z= f(x,y) 


FIGURE 12.7 A surface S with an explicit representation, z = f(x, y). The region T 
is the projection of S on the xy-plane. 


+ Since the integral in (12.7) involves r, the area of a surface will depend on the function used to describe 
the surface. When we discuss surface integrals we shall prove (in Section 12.8) that under certain general 
conditions the area is independent of the parametric representation. The result is analogous to Theorem 
10.1, in which we discussed the invariance of line integrals under a change of parameter. 
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When S lies in a plane parallel to the xy-plane, the function f is constant, so 0f/dx = 
Of/ey = 0, and Equation (12.9) becomes 


a(S) = {| dx dy. 
T 


This agrees with the usual formula for areas of plane regions. 

Equation (12.9) can be written in another form that gives further insight into its geometric 
significance. At each point of S, let y denote the angle between the normal vector 
N = Or/0x x Or/Oy and the unit coordinate vector k. (See Figure 12.8.) Since the z- 
component of N is 1, we have 


INIA LN Or Sor 


Ox Oy 


and hence |\0r/0x x Or/dy|| = 1/cosy. Therefore Equation (12.9) becomes 


cos y 


(12.10) a(S) = {{ : dx dy. 
T 
Suppose now that S lies in a plane not perpendicular to the xy-plane. Then y is constant 
and Equation (12.10) states that the area of S = (area of T)/cos y, or that 
(12.11) a(T) = a(S) cos y. 
Equation (12.11) is sometimes referred to as the area cosine principle. It tells us that if 


a region S in one plane is projected onto a region T in another plane, making an angle y 
with the first plane, the area of T is cos y times that of S. This formula is obviously true 


x acos y y 


FIGURE 12.9 The area cosine principle for 


or Or | 
FIGURE 12.8 The length of x ay 1S 1/cos y. a rectangle. 
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when S is the rectangle shown in Figure 12.9, because distances in one direction are 
shortened by the factor cos y while those in a perpendicular direction are unaltered by 
projection. Equation (12.11) extends this property to any plane region S having an area. 

Suppose now that S is given by an implicit representation F(x, y,z) =0. If S can be 
projected in a one-to-one fashion on the xy-plane, the equation F(x, y, z) = 0 defines z 
as a function of x and y, say z = f(x, y), and the partial derivatives df/0x and df/dy are 
related to those of F by the equations 


Of OF |x 4 Of __—*OF/ey 
Ox OF /dz Oy OF /dz 


for those points at which 0F/dz 4 0. Substituting these quotients in (12.9), we find 


2 2 2 
(12.12) a(S) = | V @F 8x)" + (GF [ayy + (GF /@z)° dx dy. 
|OF /dz| 
T 
EXAMPLE 1. Area of a hemisphere. Consider a hemisphere S of radius a and center at 
the origin. We have at our disposal the implicit representation x? + y? + 2 =a?,z>0; 


the explicit representation z = Jat — x2 — y”; and the parametric representation 
(12.13) r(u, v) = acosucosvi+asinucosvj+asinvk. 


To compute the area of S from the implicit representation we refer to Equation (12.12) 
with 
F(x,y2J=eVP+y+2—a’. 


The partial derivatives of F are 0F/0x = 2x, OF/0y = 2y, OF/0z = 2z. The hemisphere S 
projects in a one-to-one fashion onto the circular disk D = {(x, y) | x? + y? < a®} in 
the xy-plane. We cannot apply Equation (12.12) directly because the partial derivative 
OF/dz is zero on the boundary of D. However, the derivative 0F/0z is nonzero everywhere 
in the interior of D, so we can consider the smaller concentric disk D(R) of radius R, 
where R < a. If S(R) denotes the corresponding portion of the upper hemisphere, Equa- 
tion (12.12) is now applicable and we find 


area of S(R) - | V (2x)? + (2y)* + 22)" 5, dy 


[2z| 
D(R) 
vA 
D(R) D(R) Va? — x? — y 


The last integral can be easily evaluated by the use of polar coordinates, giving us 


area of S(R) = a ‘ea r ar dé = 2na(a — Ja? — R?), 


Qe 


When R — a this approaches the limit 27ra?. 
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We can avoid the limiting process in the foregoing calculation by using the parametric 
representation in (12.13). The calculations of Example 2 in Section 12.2 show that 


Or or 
a x eens: 
Ou av 


= |la cos vr(u, v)|| = a? |cos v}. 


Therefore we can apply Equation (12.7), taking for the region 7 the rectangle [0, 27] x 
[0, dar]. We find 


a(S) = a {| lcos v| du dv = a \e | [°cos v do | du = 27a". 
T 


EXAMPLE 2. Another theorem of Pappus. One of the theorems of Pappus states that a 
surface of revolution, obtained by rotating a plane curve of length Z about an axis in the 
plane of the curve, has area 27Lh, where h is the distance from the centroid of the curve 
to the axis of rotation. We shall use Equation (12.7) to prove this theorem. 


7 


z= f(x) ae 
XYZ 


- _f 
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x 
Ficure 12.10 Area of a surface of revolution determined by Pappus’ theorem. 
Suppose a curve C, initially in the xz-plane, is rotated about the z-axis. Let its equation 


in the xz-plane be z = f(x), where a<x<b,a>0. The surface of revolution S so 
generated can be described by the vector equation 


r(u,v) =ucosvi+usinvj + fk, 
where (u, v) € [a, b] x [0, 27]. The parameters u and v can be interpreted as the radius 


and angle of polar coordinates, as illustrated in Figure 12.10. If a<u< J, all points 
(x,-y, z) at a given distance u from the z-axis have the same z-coordinate, f(u), so they 
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all lie on the surface. The fundamental vector product of this representation is 


i j k 
Or or ; ; : soe soc 
rea ie COS D sinv f'(u)| = —uf'(u) cos vi — uf’(u) sin vj + uk, 
u v 
—usinv ucosv’ QO 
and hence 
Or or So eee 
x Suvi + wr. 
Ou adv 


Therefore Equation (12.7) becomes 
a(S) = i ik uv +O f'(wP du | dv = 27 fi: uJi + [f’(u)}? du. 


The last integral can be expressed as f¢ x ds, a line integral with respect to arc length along 
the curve C. As such, it is equal to xL, where x is the x-coordinate of the centroid of C and 
L is the length of C. (See Section 10.8.) Therefore the area of S is 27LxX. This proves the 
theorem of Pappus. 


12.6 Exercises 


1. Let S be a parallelogram not parallel to any of the coordinate planes. Let S,, S,, and S 
denote the areas of the projections of S on the three coordinate planes. Show that the area of S 
is \/S? + S23 + S2. 

2. Compute the area of the region cut from the plane x + y +z =a by the cylinder 
e+ yt =a’, 

3. Compute the surface area of that portion of the sphere x? + y? + z? = a? lying within the 
cylinder x? + y* = ay, wherea > 0. 

4. Compute the area of that portion of the surface z? = 2xy which lies above the first quadrant 
of the xy-plane and is cut off by the planes x = 2andy = 1. 

5. A parametric surface S is described by the vector equation 


r(u,v) =ucosvi +tusinvj + uk, 


where 0 <u <4and0 <v < 27. 

(a) Show that S is a portion of a surface of revolution. Make a sketch and indicate the 
geometric meanings of the parameters u and v on the surface. 

(b) Compute the fundamental vector product or/du x or/dv in terms of u and v. 


(c) The area of Sis 7(654/ 65 — 1)/n, where x is an integer. Compute the value of n. 

6. Compute the area of that portion of the conical surface x? + y? = z* which lies above the 
xy-plane and is cut off by the sphere x? + y? + z? = 2ax. 

7. Compute the area of that portion of the conical surface x” + y* = z® which lies between the 
two planes z = Oand x + 2z =3. 

8. Compute the area of that portion of the paraboloid x? + z* = 2ay which is cut off by the 
plane y =a. 

9. Compute the area of the torus described by the vector equation 


r(u,v) = (a + bcosu)sinvi + (a+ bcosu)cosvj + bsinuk, 


where 0 <b <aand0 <u < 27,0 <v < 27. [Hint: Use the theorem of Pappus.] 
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10. A sphere is inscribed in a right circular cylinder. The sphere is sliced by two parallel planes 
perpendicular to the axis of the cylinder. Show that the portions of the sphere and cylinder 
lying between these planes have equal surface areas. 

11. Let T be the unit disk in the uv-plane, T = {(u, v) | u? + v? < 1}, and let 


Gee 2u ; 2v pe es 
ieee eee eee? u + vy + | 

(a) Determine the image of each of the following sets under r: the unit circle u? + v? = 1; 
the interval -1 <u <1; that part of the line uv = v lying in T. 

(b) The surface S = r(T) is a familiar surface. Name and sketch it. 


(c) Determine the image of the uv-plane under r. Indicate by a sketch in the xyz-space the 
geometric meanings of the parameters u and v. 


12.7 Surface integrals 


Surface integrals are, in many respects, analogous to line integrals; the integration takes 
place along a surface rather than along a curve. We defined line integrals in terms of a 
parametric representation for the curve. Similarly, we shall define surface integrals in 
terms of a parametric representation for the surface. Then we shall prove that under 
certain general conditions the value of the integral is independent of the representation. 


DEFINITION OF A SURFACE INTEGRAL. Let S =r(T) be a parametric surface described 
by a differentiable function r defined on a region T in the uv-plane, and let f be a scalar field 
defined and bounded on S. The surface integral of f over S is denoted by the symbol J{ f dS 


[or by JJ f(x, y, z) dS], and is defined by the equation (7) 
Ss 
(12.14) {|r dS = J) sec v)] du dv 
r(T) 


whenever the double integral on the right exists. 


The following examples illustrate some applications of surface integrals. 


EXAMPLE |. Surface area. When f = 1, Equation (12.14) becomes 


dS = 


du dv. 


The double integral on the right is that used earlier in Section 12.5 to define surface area. 

Thus, the area of S is equal to the surface integral {jf dS. For this reason, the symbol 
r(T) 

dS its sometimes referred to as an “element of surface area,’ and the surface integral 

Sf f dS is said to be an integral of f with respect to the element of surface area, extended 

r(T) 

over the surface r(T). 
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EXAMPLE 2. Center of mass. Moment of inertia. If the scalar field f is interpreted as 
the density (mass per unit area) of a thin material in the shape of the surface S, the total 
mass m of the surface is defined by the equation 


m= ie y, z) dS. 


S 


Its center of mass is the point (x, y, Z) determined by the equations 
sm=||xf@y.zdS, gm= |] vf@y2dS, zm=]J f(x,y, 2) dS. 
Ss Ss Ss 
The moment of inertia J, of S about an axis L is defined by the equation 


1, = || %x, y, Df y, 2) dS, 
S 


where 6(x, y, z) denotes the perpendicular distance from a general point (x, y,z) of S 
to the line L. 

To illustrate, let us determine the center of mass of a uniform hemispherical surface of 
radius a. We use the parametric representation 


r(u,v) = acosucosvi+asinucosvj+asinvk, 


where (u, v) € [0, 27] x [0, $7]. This particular representation was discussed earlier in 
Example 2 of Section 12.2, where we found that the magnitude of the fundamental vector 
product is a*|cos v|. In this example the density fis constant, say f= c, and the mass 
m is 27ra*c, the area of S times c. Because of symmetry, the coordinates x and y of the 
center of mass are 0. The coordinate Z is given by 


tm=c||zdS=c]| asinv-a®|coso| du do 
Ss T 


7/2 
a 
= 2nate | sin v cos v dv = 7ra®c = 5 
0 


sozZ=a/2. 


EXAMPLE 3, Fluid flow through a surface. We consider a fluid as a collection of points 
called particles. At each particle (x, y, z) we attach a vector V(x, y, z) which represents the 
velocity of that particular particle. This is the velocity field of the flow. The velocity field 
may or may not change with time. We shall consider only steady-state flows, that is, flows 
for which the velocity V(x, y, z) depends only on the position of the particle and not on 
time. 

We denote by p(x, y, z) the density (mass per unit volume) of the fluid at the point 
(x,y,z). If the fluid is incompressible the density p will be constant throughout the fluid. 
For a compressible fluid, such as a gas, the density may vary from point to point. In any 
case, the density is a scalar field associated with the flow. The product of the density and 
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the velocity we denote by F; that is, 


F(x, y, Z) = p(x, y, Z)V (x, y, Z). 


This is a vector field called the flux density of the flow. The vector F(x, y, z) has the same 
direction as the velocity, and its length has the dimensions 


mass distance mass 


unit volume unit time — (unit area)(unit time) 


In other words, the flux density vector F(x, y, z) tells us how much mass of fluid per unit 
area per unit time is flowing in the direction of V(x, y, z) at the point (x, y, z). 

Let S = r(T) be a simple parametric surface. At each regular point of S let m denote the 
unit normal having the same direction as the fundamental vector product. That is, let 


Or or 
ames x pean 
Ou Ov 
Or Or 
—— x pases 
Ou Ov 


(12.15) i= 


The dot product F- represents the component of the flux density vector in the direction 
of nm. The mass of fluid flowing through S in unit time in the direction of n is defined to be 


the surface integral 
[[F-nas = [[ eon 


r(T) T 


Or or 
a= S61 
Ou av 


du dv. 


12.8 Change of parametric representation 


We turn now to a discussion of the independence of surface integrals under a change 
of parametric representation. Suppose a function r maps a region A in the uv-plane onto 
a parametric surface r(A). Suppose also that A is the image of a region B in the st-plane 
under a one-to-one continuously differentiable mapping G given by 


(12.16) G(s, t) = U(s, i+ V(s, thy if (s,reB. 
Consider the function R defined on B by the equation 
(12.17) R(s, t) = r[G(s, 0]. 


(See Figure 12.11.) Two functions r and R so related will be called smoothly equivalent. 
Smoothly equivalent functions describe the same surface. That is, r(A) and R(B) are 
identical as point sets. (This follows at once from the one-to-one nature of G.) The next 
theorem describes the relationship between their fundamental vector products. 
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FiGureE 12.11 Two parametric representations of the same surface. 


THEOREM 12.1. Let rand R be smoothly equivalent functions related by Equation (12.17), 
where G = Ui + Vj is a one-to-one continuously differentiable mapping of a region B in the 
st-plane onto a region A in the uv-plane given by Equation (12.16). Then we have 


OR OR Or _ or\a(U, V) 
12.18 — x — = (— x —|——_--,, 
( ) Os ° ot i . d O(s, t) 


where the partial derivatives 0r/du and Or] dv are to be evaluated at the point (U(s, t), V(s, t)). 
In other words, the fundamental vector product of R is equal to that of r, times the Jacobian 
determinant of the mapping G. 


Proof. The derivatives 0R/ds and 0R/dt can be computed by differentiation of Equation 
(12.17). If we apply the chain rule (Theorem 8.8) to each component of R and rearrange 
terms, we find that 


OR _ a aU, av |, OR_ OU, ar av 
Os oudos dvds Ot Oudot  dvdat- 


where the derivatives Or/du and Or/dv are evaluated at (U(s, t), V(s, t)). Now we cross 
multiply these two equations and, noting the order of the factors, we obtain 


Os ot Ot Os 


OR . OR _ S . een ea _ (F % a V) 
Os ot Ou Ov du dv) As, t) — 


This completes the proof. 


The invariance of surface integrals under smoothly equivalent parametric representa- 
tions is now an easy consequence of Theorem 12.1. 
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THEOREM 12.2. Let r and R be smoothly equivalent functions, as described in Theorem 
12.1. If the surface integral J J es dS exists, the surface integral F) J t dS also exists and we 


have 
[| ras =|] fas. 
r(A4) R(B) 
Proof. By the definition of a surface integral we have 


|] sas= J Ffir(u, v)] 


r( A) 


oF 


—|\du dv. 
=a 


Now we use the mapping G of Theorem 12.1 to transform this into a double integral over 
the region B in the st-plane. The transformation formula for double integrals states that 


JJ Fee ) 


where the derivatives dr/du and Or/dv on the right are to be evaluated at (U(s, t), V(s, £)). 
Because of Equation (12.18), the integral over B is equal to 


[frm | 


This, in turn, is the definition of the surface integral Jf fdS. The proof is now complete. 
R(B) 


o(U, V) 
A(s, t) 


dudv= ds at, 


12.9 Other notations for surface integrals 


If S = r(7) is a parametric surface, the fundamental vector product N = dr/du x dr/dv 
is normal to S at each regular point of the surface. At each such point there are two unit 
normals, a unit normal m, which has the same direction as N, and a unit normal m, which 
has the opposite direction. Thus, 


N 
ny ae and Ns — —n,. 


| NII 


Let a be one of the.two normals m, or n,. Let F be a vector field defined on S and assume 
the surface integral {{ F- dS exists. Then we can write 
S 


sian [| #-nas = { [Ftece, 091 mu, 0) — x= 
§ T 


= «|| Free x Lae dv 
é Ou Ov 


where the + sign is used if m = nm, and the — sign is used ifm = ny. 


du dv 
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Suppose now we express F and r in terms of their components, say 


F(x, y, Zz) = PO, y, z)i + Q(x, y, zi + RO, y, z)k 
and 
r(u, v) = X(u, vi + Yu, v)j + Z(u, v)k. 


Then the fundamental vector product of r is given by 


y= 2 or _ YZ), ZX), AK Y), 
u 


dv O(u, v) O(u, v) O(u, v) 


If n = n,, Equation (12.19) becomes 


(12.20) J) F-ndS = J) Po v)] Pant dv 


+ | | Ofr(u, v)] So du do + | | Rir(u, v)] Go du do 


if m = n,, each double integral on the right must be replaced by its negative. The sum of 
the double integrals on the right is often written more briefly as 


(12.21) {| P(x, y,z)dy Adz + {| O(x, y,z)dz Adx + {| R(x, y, z)dx Ady, 
Ss Ss Ss 


or even more briefly as 


(12.22) [| Pay adz+Qdzndx+Rdx Ady. 
Ss 


The integrals which appear in (12.21) and (12.22) are also referred to as surface integrals. 
Thus, for example, the surface integral [J P dy A dz is defined by the equation 
Ss 


(12.23) [p tinae= {| Prana jade. 
O(u, v) 
Ss T 
This notation is suggested by the formula for changing variables in a double integral. 
Despite similarity in notation, the integral on the left of (12.23) is not a double integral. 
First of all, P is a function of three variables. Also, we must take into consideration the 
order in which the symbols dy and dz appear in the surface integral, because 


O(Y,Z) _ _ O(Z, Y) 
O(u, v) 7 O(u, v) 


and hence 


[| Pdyndz=—|| Pdz aay. 
S S 
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In this notation, formula (12.20) becomes 


(12.24) | Fenas = {[ Pdyadz + Qdzadx+Rdx Ady 
Ss iS 


ifm = n,. If m = ny, the integral on the right must be replaced by its negative. This formula 
resembles the following formula for line integrals: 


| F-da=| Pax +Qdy +Radz. 


If the unit normal a is expressed in terms of its direction cosines, say 
n=cosai+cosfj+cosyk, 


then F-n = Pcosa + Qcosf + Rceosy, and we can write 


[[ r-nds = || (Pcosa + QcosB + Roos y) dS. 
iS S 


This equation holds when a is either m, or m,. The direction cosines will depend on the 
choice of the normal. If m = m, we can use (12.24) to write 


(12.25) 


[| (Pcosa + Q cos B+ Roos y) dS = {{ Pdy Adz + Qdzadx + Rdx Ady. 
Ss iS 


If nm = n, we have, instead, 


(12.26) 


[| (Pcosa + Q cos B + Reos y) dS = —|| Pdy Adz + QdzAdx + Rdx Ady. 
S S 


12.10 Exercises 


1. Let S denote the hemisphere x? + y? + z2? =1, z >0, and let F(x, y,z) = xi + yj. Let 
n be the unit outward normal of S. Compute the value of the surface integral Jj F - nds, 
using : S 
(a) the vector representation r(u, v) = sinucosvi + sinusinvj + cosuk, 

(b) the explicit representation z = ./1 — x? — y®. 

2. Show that the moment of inertia of a homogeneous spherical shell about a diameter is equal to 
2ma*, where m is the mass of the shell and a is its radius. 

3. Find the center of mass of that portion of the homogeneous hemispherical surface x? + y? + 
z® = qa” lying above the first quadrant in the xy-plane. 


10. 
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. Let S denote the plane surface whose boundary is the triangle with vertices at (1, 0, 0), (0, 1, 0), 


and (0,0, 1), and let F(x, y, z) = xi + yj + zk. Let m denote the unit normal to S having 
a nonnegative z-component. Evaluate the surface integral Jj F - n dS, using: 


(a) the vector representation r(u, v) = (u + v)i+ (u — vj + (1 — 2Wk, 
(b) an explicit representation of the form z = f(x, y). 


. Let S be a parametric surface described by the explicit formula z = f(x, y), where (x, y) varies 


over a plane region 7, the projection of S in the xy-plane. Let F = Pi + Qj + Rk and let 
n denote the unit normal to S having a nonnegative z-component. Use the parametric re- 
presentation r(x, y) = xi + yj + f(x, y)k and show that 


\Jr nds = II(- -P£ - OF +R) dedy, 


where each of P, Q, and R is to be evaluated at (x, y, f(x, y)). 


. Let S be as in Exercise 5, and let g be a scalar field. Show that: 


2 an zy 
(a) P(x, y,z)dS = pix, y, f(x, y)] jl + ax dx dy. 
S T 


of 
-|{ Pix, yi f(x, y)] ax dx dy. 
T 


(b) {| p(x, y, z) dy Adz 
S 


a 
(c) {| Q(x, y, z) dz A dx -{| px, y, f(x, y)] z dx dy. 
S T 


. If S is the surface of the sphere x? + y* + z? = a?, compute the value of the surface integral 


| [ xzay A dz + yzdz ndx + x*dx andy. 
Ss 


Choose a representation for which the fundamental vector product points in the direction of the 
outward normal. 


. The ey nee x? + y® = 2x cuts out a portion of a surface S from the upper nappe of the cone 


x? + y® = z*, Compute the value of the surface integral 


i (x* — y* + y?z? — 22x? + 1) dS. 
Ss 


. A homogeneous spherical shell of radius a is cut by one nappe of a right circular cone whose 


vertex is at the center of the sphere. If the vertex angle of the cone is «, where0 <« <7, 
determine (in terms of a and «) the center of mass of the portion of the spherical shell that lies 
inside the cone. 

A homogeneous paper rectangle of base 27a and altitude / is rolled to form a circular cylindrical 
surface S of radius a. Calculate the moment of inertia of S about an axis through a diameter 
of the circular base. 
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11. Refer to Exercise 10. Calculate the moment of inertia of S about an axis which is in the plane 
of the base and is tangent to the circular edge of the base. 

12. A fluid flow has flux density vector F(x, y, z) = xi — (2x + y)j + zk. Let S denote the hemi- 
sphere x? + y? + z2 = 1,z > 0, and let v denote the unit normal that points out of the sphere. 
Calculate the mass of fluid flowing through S in unit time in the direction of a. 

13. Solve Exercise 12 if S also includes the planar base of the hemisphere. On the lower base the 
unit normal is —k. 

14. Let S denote the portion of the plane x + y +z = 1 cut off by the unit sphere x? + y? + 
z2=1. Let o(x,y,z) =1 — x* — y* — z? if (x, y, z) is inside this sphere, and let (x, y, z) 
be 0 otherwise. Show that 


(3-722 if |t] < /3, 
{| P(x, y,z) dS = 18 we 
8 0 if |t| > /3. 


[Hint: Introduce new coordinates (x,, y,, Z,) with the z,-axis normal to the plane 
x+y +z =t. Theuse the polar coordinates in the x, y,-plane as parameters for S.] 


12.11 The theorem of Stokes 


The rest of this chapter is devoted primarily to two generalizations of the second funda- 
mental theorem of calculus involving surface integrals. They are known, respectively, as 
Stokes’ theorem} and the divergence theorem. This section treats Stokes’ theorem. The 
divergence theorem is discussed in Section 12.19. 

Stoke’s theorem is a direct extension of Green’s theorem which states that 


[J (2-2) avay = f pax + ody, 
: Ox oy C 


where S is a plane region bounded by a simple closed curve C traversed in the positive 
(counterclockwise) direction. Stokes’ theorem relates a surface integral to a line integral 
and can be stated as follows. 


THEOREM 12.3 STOKES’ THEOREM. Assume that S is a smooth simple parametric surface, 
say S=r(T), where T is a region in the uv-plane bounded by a piecewise smooth Jordan 
curve I’. (See Figure 12.12.) Assume also that r is a one-to-one mapping whose components 
have continuous second-order partial derivatives on some open set containing TUT. Let C 
denote the image of I under r, and let P, Q, and R be continuously differentiable scalar fields 
on S. Then we have 


(12.27) |} (F-2) aynacs (F-B) ar nars C-%) axaay 
JJ \ay ~ az ae ax: Oy 


Z x 
-| Pdx+Qdy+Rdz. 
C 


ft In honor of G. G. Stokes (1819-1903), an Irish mathematician who made many fundamental contributions 
to hydrodynamics and optics. 
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x 


FiGureE 12.12 An example of a surface to which Stokes’ theorem is applicable. 


The curve I is traversed in the positive (counterclockwise) direction and the curve C is traversed 
in the direction inherited from T through the mapping function r. 


Proof. To prove the theorem it suffices to establish the following three formulas, 


(12.28) [parm |J (—2acnay + ae nas), 
C (p Oy Oz 
foa=JJ (— So ay adz + 2 dxady), 
C Oz Ox 


[aca JJ (-Ba A dx + 5 dy naz). 


Addition of these three equations gives the formula (12.27) in Stokes’ theorem. Since the 
three are similar, we prove only Equation (12.28). 

The plan of the proof is to express the surface integral as a double integral over 7. Then 
we use Green’s theorem to express the double integral over 7 as a line integral over I’. 
Finally, we show that this line integral is equal to fg P dx. 

We write 


r(u,v) = X(u, vi + Y(u, v)j + Z(u, v)k 


and express the surface integral over S in the form 


Jf (— Barn ays Banas) = [f{- LAD 4 BAL) ana 
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Now let p denote the composite function given by 
plu, v) = P[X(u, v), Y(u, v), Zu, v)). 


The last integrand can be written as 


(12.29) see =| o*) = s.(0 5) 
Oy Ou,v) Oz Au,v) du\ av Ov\ du 


The verification of (12.29) is outlined in Exercise i3 of Section 12.13. Applying Green’s 
theorem to the double integral over T we obtain 


0 ( Ox 0 (ox OX OX 
—_ —_—|—-—-— aes du dv = — du + = du; 
J) i (? ) sil? au ) rau Op 
where I’ is traversed in the positive direction. We parametrize I by a function y defined on 
an interval [a, b] and let 
(12.30) a(t) = rly(t)] 


be a corresponding parametrization of C. Then by expressing each line integral in terms 
of its parametric representation we find that 


po au + p& do = | Pax, 
r Ou Ov C 


which completes the proof of (12.28). 


12.12 The curl and divergence of a vector field 


The surface integral which appears in Stokes’ theorem can be expressed more simply in 
terms of the curl of a vector field. Let F be a differentiable vector field given by 


F(x, y, 2) = P(x, y, 26 + O(n, y, 2 + ROX y, DE. 


The curl of F is another vector field defined by the equation 


dR 00\.. (@P @R\.. (00 2) 
—— Pod : a i be x) oy es ay 


The components of curl F are the functions appearing in the surface integral in Stokes’ 
formula (12.27). Therefore, this surface integral can be written as 


{{ (url F)- nds, 
S 
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where n is the unit normal vector having the same direction as the fundamental vector 
product of the surface; that is, 

Or _ or 

a xX — 

Ou ov 

du av 


j= 


The line integral in Stokes’ formula (12.27) can be written as {¢ F: da, where @ is the 
representation of C given by (12.30). Thus, Stokes’ theorem takes the simpler form 


J (curl F)-ndS = ie da. 


For the special case in which S is a region in the xy-plane and a = k, this formula reduces 


to Green’s theorem, 
JJ @- sr) dx dy = | Pax + Ody. 
Ox oy C 


Equation (12.31) defining the curl can be easily remembered by writing it as an expansion 
of a3 x 3 determinant, 


i ej ik 
0 0 @ OR a0 OP OR\. 00 OP 
ee aie Se oe | 
oes Ox Oy az = (; y Oz ji es a é ce 0 ) 
P OR 


This determinant is to be expanded in terms of first-row minors, but each “‘product”’ such 
as 0/Oy times R is to be interpreted as a partial derivative 0R/dy. We can also write this 
formula as a cross product, 


cullF=V x F, 


where the symbol V is treated as though it were a vector, 


If we form the “‘dot product” V- F in a purely formal way, again interpreting products 
such as 0/0x times P as OP/0x, we find that 


oP o0Q 
12.32 V-F=—4+— 
Ox " Oy i = 


Equation (12.32) defines a scalar field called the divergence of F, also written as div F. 
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We have already used the symbol V@ to denote the gradient of a scalar field ~, given by 


This formula can be interpreted as formal multiplication of the symbolic vector V by the 
scalar field y. Thus, the gradient, the divergence, and the curl can be represented sym- 
bolically by the three products Vy, V- F, and V x F, respectively. 

Some of the theorems proved earlier can be expressed in terms of the curl. For example, 
in Theorem 10.9 we proved that a vector field f= (f,,...,/,,), continuously differentiable 
on an open convex set S in n-space, is a gradient on Sif and only if the partial derivatives of 
the components of f satisfy the relations 


(12.33) D, f(x) = D,fAx) (j,k =1,2,...,7). 


In the 3-dimensional case Theorem 10.9 can be restated as follows. 


THEOREM 12.4. Let F = Pi + Qj + Rk be a continuously differentiable vector field on an 
open convex set S in 3-space. Then F is a gradient on S if and only if we have 


(12.34) culF=O on S. 


Proof. \n the 3-dimensional case the relations (12.33) are equivalent to the statement 
that curl F= O. 


12.13 Exercises 
In each of Exercises 1 through 4, transform the surface integral {f (curl F) - dS to a line integral 
iS 
by the use of Stokes’ theorem, and then evaluate the line integral. 


1. F(x, y,z) = y"i + xyj + xzk, where S is the hemisphere x? + y? +z? =1, z>0, and n is 
the unit normal with a nonnegative z-component. 

2. F(x, y, Zz) = yi + 2j + xk, where S is the portion of the paraboloid z = 1 — x? — y? with 
z > 0, and wv is the unit normal with a nonnegative z-component. 

3. F(x, y,z) = (y — z)i + yzj — xzk, where S consists of the five faces of the cube 0 < x <2, 
0O<y <2,0 <¢z <2 not in the xy-plane. The unit normal z is the outward normal. 

4. F(x, y,z) = xzi — yi + x*yk, where S consists of the three faces not in the xz-plane of the 
tetrahedron bounded by the three coordinate planes and the plane 3x + y + 3z =6. The 
normal n is the unit normal! pointing out of the tetrahedron. 


In Exercises 5 through 10, use Stokes’ theorem to show that the line integrals have the values 
given. In each case, explain how to traverse C to arrive at the given answer. 


5. Jo ydx +zdy + x dz = na®,/3, where C is the curve of intersection of the sphere x? + 
y? +z% =a’ and the planex +y +z =0. 

6. Jo (y +z) dx + (z + x) dy + (x + y) dz =0, where C is the curve of intersection of the 
cylinder x? + y? = 2y and the plane y = z. 

7. Je y® dx + xy dy + xz dz =0, where C is the curve of Exercise 6. 
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8. fa ly — z)dx + (z — x) dy + (x — y) dz =2za(a + b), where C is the intersection of the 
cylinder x? + y* = a? and the plane x/a + z/b = 1,a>0,56>0. 

9. fo (Gy? +27) dx + (x? + 27) dy + (x? + y®) dz = 2nab*, where C is the intersection of the 
hemisphere x? + y? + z? = 2ax, z > 0, and the cylinder x? + y? = 2bx, whereO0 <b <a. 

10. fo (Gy? — 27) dx + (z? — x*) dy + (x? — y*) dz = 9a*/2, where C is the curve cut from the 
boundary of the cubeO <x <a,0<y <a,0<z<abythe planex + y +z = 3a/2. 

11. If r = xi + yj + zk and Pi + Qj + Rk =a Xr, where a is a constant vector, show that 
Sa Pdx + Qdy + R dz =2Sja-ndS, where C is a curve bounding a parametric surface S 


S 
and n is a suitable normal to S. 

12. Let F = Pi + Qj + Rk, where P = —y/(x? + y*), QO = x/(x? + y?), R =z, and let D be 
the torus generated by rotating the circle (x — 2)? +227 =1, y =0, about the z-axis. Show 
that curl F = O in D but that {> Pdx + Qdy + Rdz is not zero if the curve C is the circle 
x+y =4,z=0. 

13. This exercise outlines a proof of Equation (12.29), used in the proof of Stokes’ theorem. 

(a) Use the formula for differentiating a product to show that 


*( ox) ( o) ap @X ap ax 


au\? av) av\P du) ~~ du dv dv Ou 


(b) Now let p(u, v) = P[X(u, v), Y(u,v), Z(u, v)]. Compute ap/eu and dp/av by the chain 
rule and use part (a) to deduce Equation (12.29), 

a/ ax a( eX\ = aP AX, Y) AP a(Z, X) 

aul? av} wi? au} — dy a(u, v) eee A(u, v) 


12.14 Further properties of the curl and divergence 


The curl and divergence of a vector field are related to the Jacobian matrix. If F = 
Pi + Qj+ Rk, the Jacobian matrix of F is 


oP oP oP 
Ox Oy az 
00 oQ ad 
DF(x, y, Zz) = ax A 
OR OR OR 
Ox Oy Oz_ 


The trace of this matrix (the sum of its diagonal elements) is the divergence of F. 

Every real matrix A can be written as a sum of a symmetric matrix, $(A + A‘), anda 
skew-symmetric matrix, 3(A — A‘). When A is the Jacobian matrix DF, the skew-symmetric 
part becomes 


0P @Q @P_ OR 


0 la 
Oy Ox Oz Ox 
1| 0Q OP 0Q OR 
12.3 -| -—-— 0 
nen 2) 0x oy Oz Oy 
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The nonzero elements of this matrix are the component of curl F and their negatives. If the 
Jacobian matrix DF is symmetric, each entry in (12.35) is zero and curl F = O. 


EXAMPLE 1. Let F(x, y, z) = xi + yj + zk. Then we have 
P(x, y,zZ)=x, O(x,y,z)=y, R(x, y, z) =Z, 
and the corresponding Jacobian matrix is the 3 x 3 identity matrix. Therefore 
div F = 3 and curl F= O. 


More generally, if F(x, y, z) = f(x)i + g(y)j + A(z)k, the Jacobian matrix has the elements 
Tt’), g’(y), A’(z) on the main diagonal and zeros elsewhere, so 


div F = f’(x) + g’(y) + A(z) and curl F= O. 
EXAMPLE 2, Let F(x, y, z) = xy?z4i + 2? sin yj + x*e%k. The Jacobian matrix is 


yz? 2xyz® — Axy?z 
0 <zcosy 2zsiny|]. 
2xev = x ev 0 
Therefore, 
div F = y*z? + z? cosy 
and 
curl F = (x?e¥ — 2z sin y)i + (2xy?z — 2xe")j — 2xyz*k. 


EXAMPLE 3. The divergence and curl of a gradient. Suppose F is a gradient, say 
F = grad g = 0/0xi + Oy/d0yj + Oy/0zk. The Jacobian matrix is 

ve ep OG 

Ox* Odydax dz 0x 

Ce, 29, iow 


cee Bia 28yh- Oroy 
0*@ ay Op 
Ox0z Oydz az 
Therefore, 


Op fp . oy 
dive 4. ae. 
OR RO Brg 


The expression on the right is called the Laplacian of and is often written more briefly 
as Vm. Thus, the divergence of a gradient Vq@ is the Laplacian of gy. In symbols, this is 
written 


(12.37) div (Vo) = Vo. 
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When V?o = 0, the function ¢ is called harmonic. Equation (12.37) shows that the gradient 
of a harmonic function has zero divergence. When the mixed partial derivatives in matrix 
(12.36) are continuous, the matrix is symmetric and curl F is zero. In other words, 


curl (grad y) = O 


for every scalar field q with continuous second-order mixed partial derivatives. This 
example shows that the condition curl F = O is necessary for a continuously differentiable 
vector field F to be a gradient. In other words, if curl F # O on an open set S, then F is 
not a gradient on S. We know also, from Theorem 12.4 that if curl F = O on an open 
convex set S, then F is a gradient on S. A field with zero curl is called irrotational. 


EXAMPLE 4. A vector field with zero divergence and zero curl. Let S be the set of all 
(x, y) € (0, 0), and let 
y . x 
i+ 
x24 y? x4 y? 


F(x, y) = J 

if (x, y)€ S. From Example 2 in Section 10.16 we know that F is not a gradient on S 
(although F is a gradient on every rectangle not containing the origin). The Jacobian 
matrix is 


2xy y> — x? 
(x? + y*)? (x? + y’)? 
2 _ 
(x+y) (x* + y’) 
0 0 0 


and we see at once that div F = 0 andcurl F= Oon S. 


EXAMPLE 5. The divergence and curl of a curl. If F = Pi+ Qj+ Rk, the curl of F 
is a new vector field and we can compute its divergence and curl. The Jacobian matrix of 
curl F is 

0?R eo aR a0 CR QO | 


@xdy axdz dy? dydz azdy az? 
oP @R OP BR OP OR 
Oxdz Ox* Oydz dydx dz> dzax 
Oo 8 ap Oo 8 606P COO o*P 


ax*  axdy Oydx Oy? azdx az dy_ 


If we assume that all the mixed partial derivatives are continuous, we find that 


div (curl F) = 0 
and 


(12.38) curl (curl F) = grad (div F) — V?F, 
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where V?F is defined by the equation 
V2F = (V?P)i + (V?Q)j + (V?R)k. 


The identity in (12.38) relates all four operators, gradient, curl, divergence, and Laplacian. 
The verification of (12.38) is requested in Exercise 7 of Section 12.15. 

The curl and divergence have some general properties in common with ordinary 
derivatives. First, they are /inear operators. That is, if a and 5 are constants, we have 


(12.39) div (aF + b6G)=adivF + bdivG, 
and 
(12.40) curl (aF + 6G) = acurl F + bcurlG. 


They also have a property analogous to the formula for differentiating a product: 


(12.41) div (yF) = gdivF + Vo: F, 
and 
(12.42) curl (gF) = gcurl F + Vo x F, 


where ¢ is any differentiable scalar field. These properties are immediate consequences 
of the definitions of curl and divergence; their proofs are requested in Exercise 6 of 
Section 12.15. 


If we use the symbolic vector 


0 ) 0 
V=—i+—jt+r—k 
Be Op, Oz 


once more, each of the formulas (12.41) and (12.42) takes a form which resembles more 
closely the usual rule for differentiating a product: 


V:-(9F)=9V:F+Vo-F 
and 
Vx (9F)=oVxF+V¢ x F. 


In Example 3 the Laplacian of a scalar field, V?~, was defined to be 074¢/0x? + 0?@/dy? + 
0?q/dz*. In Example 5 the Laplacian V?F of a vector field was defined by components. We 
get correct formulas for both V*y and V?F if we interpret V? as the symbolic operator 


O68 


Co" 
ae ae ee 


2 


This formula for V? also arises by dot multiplication of the symbolic vector V with itself. 
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Thus, we have V2 = V- V and we can write 
V2o = (V- V)@ and VF = (V-V)F. 


Now consider the formula V-Vq. This can be read as (V- V)y, which is V2m; or as 
V -(Vq@), which is div (V@). In Example 3 we showed that div (Vqv) = V2, so we have 


(V-V)p = V- (V9); 


hence we can write V- V@ for either of these expressions without danger of ambiguity. 
This is not true, however, when ¢ is replaced by a vector field F. The expression (V- V)F 
is VF, which has been defined. However, V- (VF) is meaningless because VF is not 
defined. Therefore the expression V- VF is meaningful only when it is interpreted as 
(V-V)F. These remarks illustrate that although symbolic formulas sometimes serve as a 
convenient notation and memory aid, care is needed in manipulating the symbols. 


12.15 Exercises 


1. For each of the following vector fields determine the Jacobian matrix and compute the curl 
and divergence. 
(a) F(x, y,z) = (x? + yz)i + (y? + xz)j + (22 + xy)k. 
(b) F(x, y, z) = (2z — 3y)i + Bx — zi + (y — 2x)k. 
(c) F(x, y, z) = (z +sin y)i — (z — xcos y)j. 
(d) F(x, y,z) = ei + cos xyj + cos xz*k. 
(ec) F(x, y,z) = x*sin yi + y* sin xzj + xy sin (cos z)k. 
_ fr =xi + y + zk andr = |r||, compute curl [f(r)r], where fis a differentiable function. 
. Ifr = xi + yy + zk and A is a constant vector, show that curl (A x r) = 2A. 
_ ifr = xi + yj + zk andr = |r|], find all integers for which div (r"r) = 0. 
. Find a vector field whose curl is xi + yj + zk or prove that no such vector field exists. 
. Prove the elementary properties of curl and divergence in Equations (12.39) through (12.42). 
. Prove that curl (curl F) = grad (div F) — V?F if the components of F have continuous mixed 
partial derivatives of second order. 
8. Prove the identity 


SINHA Hh W WN 


V-(F x G)=G:(V x F)-—F-(V x G), 


where F and G are differentiable vector fields. 

9. A vector field F will not be the gradient of a potential unless curl F = O. However, it may be 
possible to find a nonzero scalar field “ such that «F is a gradient. Prove that if such a / 
exists, Fis always perpendicular to its curl. When the field is two-dimensional, say F = Pi + Qj, 
this exercise gives us a necessary condition for the differential equation Pdx + Qdy =0 to 
have an integrating factor. (The converse is also true. That is, if F- curl F = 0 in a suitable 
region, a nonzero “ exists such that uF is a gradient. The proof of the converse is not required.) 

10. Let F(x, y, z) = y?z*i + z*x7f + x®y?k. Show that curl F is not always zero, but that 
F-curl F =0. Finda scalar field »« such that wF is a gradient. 

11. Let V(x, y) = yt + xj, where c is a positive constant, and let r(x, y) = xi + yj. Let R be 
a plane region bounded by a piecewise smooth Jordan curve C. Compute div (V x r) and 
curl (V x r), and use Green’s theorem to show that 


$ Vxr-da =0, 
C 


where «@ describes C. 
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12. Show that Green’s theorem can be expressed as follows: 


[| Gurl ~)-kdxdy =$V-Tds, 
R C 


where T is the unit tangent to C and s denotes arc length. 
13. A plane region R is bounded by a piecewise smooth Jordan curve C. The moments of inertia 
of R about the x- and y-axes are known to be aand J, respectively. Compute the line integral 


$ V(r*) -nds 


in terms ofaand b. Here r = ||xi + yj|| ,m denotes the unit outward normal of C, and s denotes 
arc length. The curve is traversed counterclockwise. 

14. Let F be a two-dimensional vector field. State a definition for the vector-valued line integral 
So F x dw. Your definition should be such that the following formula is a consequence of 
Green’s theorem: 


[ Fx da =k{{ GivF)dedy, 
C R 
where R is a plane region bounded by a simple closed curve C. 


% 12.16 Reconstruction of a vector field from its curl 


In our study of the gradient we learned how to determine whether or not a given vector 
field is a gradient. We now ask a similar question concerning the curl. Given a vector field 
F, is there a G such that curlG = F? Suppose we write F = Pi+ Qj+ Rk and G= 
Li+ Mj + Nk. To solve the equation curl G = F we must solve the system of partial 
differential equations 


(12.43) ——-—-=P, —-—=Q, —-—=R 


for the three unknown functions L, M, and N when P, Q, and R are given. 

It is not always possible to solve such a system. For example, we proved in Section 
12.14 that the divergence of a curl is always zero. Therefore, for the system (12.43) to have 
a solution in some open set S it is necessary to have 

OP  0Q ., OR 
12.44 Pa aa a 
( Ox Oy @z 
everywhere in S. As it turns out, this condition is also sufficient for system (12.43) to have 
a solution if we suitably restrict the set S in which (12.44) is satisfied. We shall prove now 
that condition (12.44) suffices when S is a three-dimensional interval. 


THEOREM 12.5 Let F be continuously differentiable on an open interval S in 3-space. 
Then there exists a vector field G such that curl G = F if and only if div F = 0 everywhere 
in S. 
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Proof. The necessity of the condition div F = 0 has already been established, since 
the divergence of a curl is always zero. To establish the sufficiency we must exhibit three 
functions L, M, and N that satisfy the three equations in (12.43). Let us try to make a 
choice with L = 0. Then the second and third equations in (12.43) become 


Oe ee and OM =. 
Ox Ox 


This means that we must have 


N(x, y,2) = —[* Olt, y, 2) dt + f0,2) 
and 


M(x, y,2) =|" R(t, y, 2) dt + gy, 2), 


where each integration is along a line segment in S and the “constants of integration” 
f(y, z) and g(y, z) are independent of x. Let us try to find a solution with f(y, z) = 0. 
The first equation in (12.43) requires 


aN _ aM 


12.45 
( ) Oy Oz 


=P. 


For the choice of M and N just described we have 


aN - 2 [ott y, 2a — 5 | RO, y,2) dt ~ 
oy xO Oz Xo Oz 


12.46 
( Oz Oy 


At this stage we interchange the two operations of partial differentiation and integration, 
using Theorem 10.8. That is, we write 


(12.47) < OG 2de= { D,O(t, y, z) dt 
Y Jxo 20 

and 

(12.48) ‘ | Reg ae= { DeR(t, y, z) dt. 
Zz 4 171) 471) 


Then Equation (12.46) becomes 


(12.49) Le { (DOU 2) = DRE Ieee: 
Oy Oz 29 Oz 


Using condition (12.44) we may replace the integrand in (12.49) by D, P(t, y, z); Equation 
(12.49) becomes 


Og 


ON C$ 
Oz 


_ OM _ [oPGnaa-% = P(x, y,Z) — P(%o, J, Z) a 
Oy Oz Xo Oz 
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Therefore (12.45) will be satisfied if we choose g so that 0g/dz = —P(x,, y,z). Thus, for 
example, we may take 


g(y, Zz) = — |? PG, yu) du. 


This argument leads us to consider the vector field G= Li+ Mj + Nk, where 
L(x, y, Zz) = 0 and 


M(x, y, Z) = i; R(t, y, z) dt — [; P(Xo, y,u) du, N(x, y, Z) = -[? Q(t, y, z) dt. 


For this choice of L, M, and N it is easy to verify, with the help of (12.47) and (12.48), 
that the three equations in (12.43) are satisfied, giving us curl G = F, as required. 


It should be noted that the foregoing proof not only establishes the existence of a vector 
field G whose curl is F, but also provides a straightforward method for determining G by 
integration involving the components of F. 

For a given F, the vector field G that we have constructed is not the only solution of the 
equation curl G = F. If we add to this G any continuously differentiable gradient Vq@ we 
obtain another solution because 


curl (G + Vg) = curl G + curl (Vg) = curl G = F, 


since curl (Vg) = O. Moreover, it is easy to show that all continuously differentiable 
solutions must be of the form G + Vq@. Indeed, if H is another solution, then curl H = 
curlG, so curl(H — G) =O. By Theorem 10.9 it follows that H — G = Vq@ for some 
continuously differentiable gradient Vy; hence H = G + Vq, as asserted. 

A vector field F for which div F = 0 is sometimes called solenoidal. Theorem 12.5 
states that a vector field is solenoidal on an open interval S' in 3-space if and only if it is the 
curl of another vector field on S. 

The following example shows that this statement is not true for arbitrary open sets. 


EXAMPLE. A solenoidal vector field that is not a curl. Let D be the portion of 3-space 
between two concentric spheres with center at the origin and radii a and 5b, where 
O<a<b. LetV=rir*®, wherer = xi + yj + zk andr = |r||. It is easy to verify that 
div V = 0 everywhere in D. In fact, we have the general formula 


div (r"r) = (n + 3)r”, 
and in this example n = —3. We shall use Stokes’ theorem to prove that this V is not a 
curl in D (although it is a curl on every open three-dimensional interval not containing the 


origin). To do this we assume there is a vector field U such that V = curl Uin D and obtain 
a contradiction. By Stokes’ theorem we can write 


(12.50) [| (curl U)- nds = > U- da, 
S 
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FiGureE 12.13 The surface S and curve C in Equation (12.50). 


where S and C are the surface and curve shown in Figure 12.13. To construct S, we take 
a spherical surface of radius R concentric with the boundaries of D, wherea< R< |), 
and we remove a small “polar cap,”’ as indicated in the figure. The portion that remains 
is the surface S. The curve C is the circular edge shown. Let n denote the unit outer normal 
of S, so that mn = r/r. Since curl U = V = r/r®, we have 


(curl U)- eee, 


ror Pr 


On the surface S this dot product has the constant value 1/R?. Therefore we have 


1 area of S 
JJ cow U)-ndS = x) 4 = Re ° 


When the polar cap shrinks to a point, the area of S approaches 47R? (the area of the 
whole sphere) and, therefore, the value of the surface integral in (12.50) approaches 47. 

Next we examine the line integral in (12.50). It 1s easy to prove that for any line integral 
fc U- da we have the inequality 


< M -Cength of C), 


[fu ae 


where M is a constant depending on U. (In fact, M can be taken to be the maximum value 
of ||U|| on C.) Therefore, as we let the polar cap shrink to a point, the length of C and the 
value of the line integral both approach zero. Thus we have a contradiction; the surface 
integral in (12.50) can be made arbitrarily close to 47, and the corresponding line integral 
to which it is equal can be made arbitrarily close to 0. Therefore a function U whose curl 
is V cannot exist in the region D. 

The difficulty here is caused by the geometric structure of the region D. Although this 
region is simply connected (that is, any simple closed curve in D is the edge of a parametric 
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surface lying completely in D) there are closed surfaces in D that are not the complete 
boundaries of solids lying entirely in D. For example, no sphere about the origin is the 
complete boundary of a solid lying entirely in D. If the region D has the property that 
every closed surface in D is the boundary of a solid lying entirely in D, it can be shown that a 
vector field U exists such that V = curl Uin D if and only if div V = 0 everywhere in D. 
The proof of this statement is difficult and will not be given here. 


%12.17 Exercises 


1. Find a vector field G(x, y, z) whose curl is 2i + j + 3k everywhere in 3-space. What is the 
most general continuously differentiable vector field with this property? 

2. Show that the vector field F(x, y,z) = (y — z)i + (z — x)j + (x — y)k is solenoidal, and 
find a vector field G such that F = curl G everywhere in 3-space. 

3. Let F(x, y,z) = —zi + xyk. Find a continuously differentiable vector field G of the form 
G(x, y,z) = L(x, y, z)i + M(x, y, z)j such that F = curl G everywhere, in 3-space. What is 
the most general G of this form? 

4. If two vector fields Uand Vare both irrotational, show that the vector field U x Vissolenoidal. 

5. Let r = xi + yj + zk and let r = ||r||. Show that n = —3 is the only value of n for which 
r"r is solenoidal for r #0. For this n, choose a 3-dimensional interval S not containing the 
origin and express r~*r as acurlin S. Note: Although r~*r is a curl in every such S, it is not 
a curl on the set of all points different from (0, 0, 0). 

6. Find the most general continuously differentiable function fof one real variable such that the 
vector field f(r)r will be solenoidal, where r = xi + yj + zk andr = |r|. 

7. Let V denote a vector field that is continuously differentiable on some open interval S in 3- 
space. Consider the following two statements about V: 

(i) curl V = Oand V =curl U for some continuously differentiable vector field U (everywhere 
on S). 
(ii) A scalar field g exists such that V @ is continuously differentiable and such that 


V = grad and V?@ =0 everywhere on S. 


(a) Prove that (i) implies (ii). In other words, a vector field that is both irrotational and 
solenoidal in S is the gradient of a harmonic function in S. 
(b) Prove that (ii) implies (i), or give a counterexample. 

8. Assume continuous differentiability of all vector fields involved, on an open interval S. Suppose 
H =F + G, where F is solenoidal and G is irrotational. Then there exists a vector field U 
such that F = curl U and a scalar field y such that G = Vqyin S. Show that U and ¢ satisfy 
the following partial differential equations in S: 


V29 =divH, — grad (div U) — V?U =curl H. 


Note: This exercise has widespread applications, because it can be shown that every 
continuously differentiable vector field H on S can be expressed in the form H = F + 
G, where F is solenoidal and G is irrotational. 


9. Let M(x, y,z) = x*yi + y’zj + z*xk. Find vector fields F and G, where F is a curl and G 
is a gradient, such thatH =F +G. 

10. Let wand t be scalar fields that are continuously differentiable on an open interval R in 3-space. 
(a) Show that a vector field F exists such that Vu x Vv = curl F everywhere in R. 
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(b) Determine whether or not any of the following three vector fields may be used for F in 
part (a): (1) V(we); (i) uwVe; (il) oVu. 
(c) Ifu(x, y,z) =x*° — yy + 2? and v(x, y,z) = x + y +z, evaluate the surface integral 


if Vu x Vo- nds, 
“s 


where S is the hemisphere x® + y? + z2 = 1, z > 0, and n is the unit normal with a non- 
negative z-component. 


12.18 Extensions of Stokes’ theorem 


Stokes’ theorem can be extended to more general simple smooth surfaces. If T is a 
multiply connected region like that shown in Figure 12.14 (with a finite number of holes), 
the one-to-one image S = r(T) will contain the same number of holes as 7. To extend 


Vv Zz 


x 


FiGurE 12.14 Extension of Stokes’ theorem to surfaces that are one-to-one images 
of multiply connected regions. 


Stokes’ theorem to such surfaces we use exactly the same type of argument as in the proof 
of Stokes’ theorem, except that we employ Green’s theorem for multiply connected regions 
(Theorem 11.12). In place of the line integral which appears in Equation (12.27) we need 
a sum Of line integrals, with appropriate signs, taken over the images of the curves forming 
the boundary of 7. For example, if T has two holes, as in Figure 12.14, and if the boundary 
curves I’, I',, and I’, are traversed in the directions shown, the identity in Stokes’ theorem 
takes the form 


|| (curl F)-ndS = $F: de + $F de, + pF dps, 
where C, C,, and C, are the images of I’, ',, and I’,, respectively, and eg, p,, and e, are 
the composite functions e(t) = r[y(t)], e1(4) = rlyi(t)], e2(t) = r[yo(t)]. Here y, y,, and 
Y2 are the functions that describe I, [',, and IT, in the directions shown. The curves C, 
C,, and C, will be traversed in the directions inherited from I’, [',, and I, through the 


mapping function r. 
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Stokes’ theorem can also be extended to some (but not all) smooth surfaces that are not 
simple. We shall illustrate a few of the possibilities with examples. 

Consider first the cylinder shown in Figure 12.15. This is the union of two simple smooth 
parametric surfaces S, and S,, the images of two adjacent rectangles 7, and 7,, under 
mappings r, and r,, respectively. If y, describes the positively oriented boundary I, of 
T, and y, describes the positively oriented boundary I, of 7,, the functions p, and Q, 
defined by 


p(t) =rlyilt)], e2(t) = relye(t)] 


describe the images C, and C, of I’, and I’,, respectively. In this example the representa- 
tions r, and r, can be chosen so that they agree on the intersection [, O [,. If we apply 


CG. 


Yr, I, 


FiGure 12.15 Extension of Stokes’ theorem to a cylinder. 


Stokes’ theorem to each piece S, and S, and add the two identities, we obtain 
(12.51) | f (curl F)-m aS + ff (curl F)- mds =|] de + | Fe des, 
Si So 1 2 


where m, and n, are the normals determined by the fundamental vector products of ry 
and r,, respectively. 

Now let r denote the mapping of 7, U T, which agrees with r,; on 7, and with r, on 
T,, and let n be the corresponding unit normal determined by the fundamental vector 
product of r. Since the normals n, and ng agree in direction on S, 1 S2, the unit normal 
nis the same as nm, on S, and the same as nm, on S,. Therefore the sum of the surface 
integrals on the left of (12.51) is equal to 


[| (curl F)- nds. 


S,USe 


For this example, the representations r, and r, can be chosen so that e, and g, determine 
opposite directions on each arc of the intersection C, M C,, as indicated by the arrows 
in Figure 12.15. The two line integrals on the right of (12.51) can be replaced by a sum 
of line integrals along the two circles C; and C, forming the upper and lower edges of 
S, U S,, since the line integrals along each arc of the intersection C, M C, cancel. There- 
fore, Equation (12.51) can be written as 


(12.52) | (curl F)- nds = | F-dp, + | F - des, 
Si VSe es ie 
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where the line integrals are traversed in the directions inherited from I, and T,. The two 
circles C; and C, are said to form the complete boundary of S, U S,. Equation (12.52) 
expresses the surface integral of (curl F):m over S, US, as a line integral over the 
complete boundary of S,; U S,. This equation is the extension of Stokes’ theorem for a 
cylinder. 

Suppose now we apply the same concepts to the surface shown in Figure 12.16. This 
surface is again the union of two smooth simple parametric surfaces S, and S,, the images 
of two adjacent rectangles 7, and T,. This particular surface is called a Mébius band:+ a 


FiGure 12.16 A Mobius band considered as the union of two simple parametric 
surfaces. Stokes’ theorem does not extend to a MObius band. 


model can easily be constructed from a long rectangular strip of paper by giving one end a 
half-twist and then fastening the two ends together. We define p,, p., C,, and C, for the 
Mobius band as we defined them for the cylinder above. The edge of S, U S, in this case is 
one simple closed curve C’, rather than two. This curve is called the complete boundary 
of the Mobius band. 

If we apply Stokes’ theorem to each piece S, and S,, as we did for the cylinder, we obtain 
Equation (12.51). But if we try to consolidate the two surface integrals and the two line 
integrals as we did above, we encounter two difficulties. First, the two normals n, and a, 
do not agree in direction everywhere on the intersection C, 1 C,. (See Figure 12.16.) 
Therefore we cannot define a normal n for the whole surface by taking m = n, on S, and 
n = n,on S,, as we did for the cylinder. This is not serious, however, because we can define 
nto be m, on S, and on C, M Cz, and then define n to be n, everywhere else. This gives a 
discontinuous normal, but the discontinuities so introduced form a set of content zero in 
the uv-plane and do not affect the existence or the value of the surface integral 


{| (curl F)- nds. 


S,USe 


A more serious difficulty is encountered when we try to consolidate the line integrals. 
In this example it is not possible to choose the mappings r, and r, in such a way that p, 
and @, determine opposite directions on each arc of the intersection C, A C,. This is 
illustrated by the arrows in Figure 12.16; one of these arcs is traced twice in the same 
direction. On this arc the corresponding line integrals will not necessarily cancel as they 


t After A. F. Mobius (1790-1868), a pupil of Gauss. At the age of 26 he was appointed professor of 
astronomy at Leipzig, a position he held until his death. He made many contributions to celestial mechanics, 
but his most important researches were in geometry and in the theory of numbers. 
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did for the cylinder. Therefore the sum of the line integrals in (12.51) is not necessarily 
equal to the line integral over the complete boundary of S, U S,, and Stokes’ theorem 
cannot be extended to the MObius band. 


Note: Thecylinder and the Mobius band are examples of orientable and nonorientable 
surfaces, respectively. We shall not attempt to define these terms precisely, but shall 
mention some of their differences. For an orientable surface S, U S, formed from two 
smooth simple parametric surfaces as described above, the mappings r, and r, can always 
be chosen so that e, and e, determine opposite directions on each arc of the intersection 
C, © Cy. Fora nonorientable surface no such choice is possible. For a smooth orientable 
surface a unit normal vector can be defined in a continuous fashion over the entire surface. 
For a nonorientable surface no such definition of a normal is possible. A paper model of 
an orientable surface always has two sides that can be distinguished by painting them with 
two different colors. Nonorientable surfaces have only one side. Fora rigorous discussion 
of these and other properties of orientable and nonorientable surfaces, see any book on 
combinatorial topology. Stokes’ theorem can be extended to orientable surfaces by a 
procedure similar to that outlined above for the cylinder. 


Another orientable surface is the sphere shown in Figure 12.17. This surface is the 
union of two simple parametric surfaces (hemispheres) S, and S,, which we may consider 


n, ent 
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images of a circular disk in the xy-plane under mappings r, and r., respectively. We give 
r, 01, Q2, Cy, Cy the same meanings as in the above examples. In this case the curves C, 
and C, are completely matched by the mapping r (they intersect along the equator), and the 
surface S; U S, is said to be closed. Moreover, r, and r, can be chosen so that the directions 
determined by pe, and pg, are opposite on C, and C,, as suggested by the arrows in Figure 
12.17. (This is why S; U S, is orientable.) If we apply Stokes’ theorem to each hemisphere 
and add the results we obtain Equation (12.51), as before. The normals m, and n, agree on 
the intersection C; \ C,, and we can consolidate the integrals over S, and S, into one 
integral over the whole sphere. The two line integrals on the right of (12.51) cancel 
completely, leaving us with the formula 


i (curl F)-ndS =0. 
SiV Se 


This holds not only for a sphere, but for any orientable closed surface. 
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12.19 The divergence theorem (Gauss’ theorem) 


Stokes’ theorem expresses a relationship between an integral extended over a surface 
and a line integral taken over the one or more curves forming the boundary of this surface. 
The divergence theorem expresses a relationship between a triple integral extended over a 
solid and a surface integral taken over the boundary of this solid. 


THEOREM 12.6. DIVERGENCE THEOREM. Let V be a solid in 3-space bounded by an orient- 
able closed surface S, and let n be the unit outer normal to S. If F isacontinuously differentiable 
vector field defined on V, we have 


(12.53) [|| (div F) dx dy dz = {{ F-nas. 
V S 


Note: If we express F and nv in terms of their components, say 


F(x, y,z) = P(x, y, z)i + Q(x, y, z)\j + R(x, y, z)k 
and 
n=cosai+cos Pj +cosyvk, 


then Equation (12.53) can be written as 
‘4 
(12.54) Je Oe ca (Pcos« + Qcosf8 + Reosy) ds. 


Proof. It suffices to establish the three equations 


J) gr av de = || Peosnds, 
[[[22eserze~ [foomeas 
V Ss 
[| Bavavaz = || reosyas, 
V Ss 


and add the results to obtain (12.54). We begin with the third of these formulas and prove 
it for solids of a very special type. 
Assume V is a set of points (x, y, z) satisfying a relation of the form 


e(x,y)<z<flx,y) for (x, y)in T, 


where J is a connected region in the xy-plane, and f and g are continuous functions on T, 
with g(x, y) < f(x, y) for each (x, y) in JT. Geometrically, this means that T is the projec- 
tion of V on the xy-plane. Every line through T parallel to the z-axis intersects the solid V 
along a line segment connecting the surface z = g(x, y) to the surface z = f(x, y). The 
boundary surface S consists of an upper cap S,, given by the explicit formula z = f(x, y) ; 
a lower part S,, given by z = g(x, y); and (possibly) a portion S3 of the cylinder generated 
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by a line moving parallel to the z-axis along the boundary of T. The outer normal to S has a 
nonnegative z-component on S,, has a nonpositive component on S,, and is parallel to the 
xy-plane on S,. Solids of this type will be called “‘xy-projectable.”’ (An example is shown 
in Figure 12.18.) They include all convex solids (for example, solid spheres, ellipsoids, 
cubes) and many solids that are not convex (for example, solid tori with axes parallel to the 
z-axis). 

The idea of the proof is quite simple. We express the triple integral as a double integral 
extended over the projection T. Then we show that this double integral has the same value 


ora oe HH HS 
so, 2 BRS 
Ge 
ee 


FiGureE 12.18 An example of a solid that is xy-projectable. 
as the surface integral in question. We begin with the formula 
f(x,y) 
{{ OR ae dy dz = |{ || OR a | dx dy. 
Oz glx.y) Oz 
V T 


The one-dimensional integral with respect to z may be evaluated by the second fundamental 
theorem of calculus, giving us 


OR 
(12.55) {{ a ee | {RIx, y, f(x, y)] — REx, y, g(x, y)]} dx dy. 
V T 
For the surface integral we can write 
(12.56) [| R cosy aS = [| R cos» dS + [[ Rosy ds + [| Ros yds. 
S Si Se Ss 


On S; the normal n is parallel to the xy-plane, so cos y = 0 and the integral over Sy is zero. 
On the surface S, we use the representation 


r(x, y) = xit yi t f(x, yk, 
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and on S, we use the representation 
r(x, y) = xi + yf + g(x, yk. 


On S, the normal a has the same direction as the vector product Or/Ox x Or/dy, so we can 
write [see Equation (12.25), p. 436] 


{| Rcos y dS = {| RdxAdy= {| R[x, y, f(x, y)] dx dy. 
Si Si T 


On S, the normal a has the direction opposite to that of dr/dx x dr/dy so, by Equation 
(12.26), we have 


[JR cosydS = _{{ RdxAdy= | R[x, y, g(x, y)] dx dy. 
Se Se T 
Therefore Equation (12.56) becomes 
{| Rcos y dS = i {RIx, y, f(x, y)] — RIx, y, g(x, y)]} dx dy. 
Ss T 


Comparing this with Equation (12.55) we see that 


[[J 9% axay az = [| Roos yas. 
a Ss 


In the foregoing proof the assumption that V is xy-projectable enabled us to express 
the triple integral over V as a double integral over its projection T in the xy-plane. It is 
clear that if V is yz-projectable we can use the same type of argument to prove the identity 


IJ gr trae = |) Pease ds: 


and if V is xz-projectable we obtain 


JI) Sp aay ae = | Qcos fds. 


Thus we see that the divergence theorem is valid for all solids projectable on all three 
coordinate planes. In particular the theorem holds for every convex solid. 


A solid torus with its axis parallel to the z-axis is xy-projectable but not xz-projectable 
or yz-projectable. To extend the divergence theorem to such a solid we cut the torus into 
four equal parts by planes through its axis parallel to the xz- and yz-planes, respectively, 
and we apply the divergence theorem to each part. The triple integral over the whole torus 
is the sum of the triple integrals over the four parts. When we add the surface integrals over 
the four parts we find that the contributions from the faces common to adjacent parts cancel 
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each other, since the outward normals have opposite directions on two such faces. There- 
fore the sum of the surface integrals over the four parts is equal to the surface integral over 
the entire torus. This example illustrates how the divergence theorem can be extended to 
certain nonconvex solids. 


12.20 Applications of the divergence theorem 


The concepts of curl and divergence of a vector field F = Pi + Qj + Rk were introduced 
in Section 12.12 by the formulas 


OP 0Q OR 
12.57 divF=—+—+4+— 
en) eS ee ay os 
and 
OR aQ OP OR 0Q OP 
12.58 lF= (—- = —= — — jk 
( a i 5.) 3 ie a - 5 | 


To compute div F and curl F from these formulas requires a knowledge of the components 
of F. These components, in turn, depend on the choice of coordinate axes in 3-space. A 
change in the position of the coordinate axes would mean a change in the components of F 
and, presumably, a corresponding change in the functions div F and curl F. With the help 
of Stokes’ theorem and the divergence theorem we can obtain formulas for the divergence 
and curl that do not involve the components of F. These formulas show that the curl and 
divergence represent intrinsic properties of the vector field F and do not depend on the 
particular choice of coordinate axes. We discuss first the formula for the divergence. 


THEOREM 12.7. Let V(t) be a solid sphere of radius t > 0 with center at a point a in 
3-space, and let S(t) denote the boundary of V(t). Let F be a vector field that is continuously 
differentiable on V(t). Then if |V(t)| denotes the volume of V(t), and if n denotes the unit 
outer normal to S, we have 


(12.59) div F(a) = lim —— fot F-nds. 
0 V(D] J 


Proof. Let y =div F. If « > 0 is given we must find a 6 > 0 such that 


(12.60) jo) — 1 |[r-nds| <« whenever 0<t <0. 
S(2) 


Since @ is continuous at a, for the given ¢ there is a 3-ball B(a; h) such that 


\@(x) — ¢(a)| < : whenever x € B(a;h). 


Therefore, if we write g(a) = ¢(x) + [y(a) — ¢(x)] and integrate both sides of this 
equation over a solid sphere V(t) of radius t < h, we find 


ota) Viol = J {J ox) dx dy az + [ff tp@ — g(x) ax dy dz. 


Vit) V(t) 
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If we apply the divergence theorem to the first triple integral on the right and then trans- 
pose this term to the left, we obtain the relation 


Ha) VO ~ |fr-nas| < ff ig@ — w(x)| dx dy dz < 5 |V(O| < € IVI. 


S(t) V(t) 


When we divide this inequality by |V(t)| we see that (12.60) holds with 6 = Ah. This proves 
the theorem. 


In the foregoing proof we made no special use of the fact that V(t) was a sphere. The 
same theorem holds true if, instead of spheres, we use any set of solids V(t) for which 
the divergence theorem is valid, provided these solids contain the point @ and shrink to 
aast—>Q. For example, each V(t) could be a cube inscribed in a sphere of radius ¢ about 
a; exactly the same proof would apply. 

Theorem 12.7 can be used to give a physical interpretation of the divergence. Suppose F 


represents the flux density vector of a steady flow. Then the surface integral Jf F-ndS 
S(t) 
measures the total mass of fluid flowing through S in unit time in the direction of n. The 


quotient fj F-ndS/|V(t)| represents the mass per unit volume that flows through S in 
St) 


unit time in the direction of nm. As t — 0, the limit of this quotient is the divergence of F at 
a. Hence the divergence at a can be interpreted as the time rate of change of mass per unit 
volume per unit time at a. 

In some books on vector analysis, Equation (12.59) is taken as the definition of diver- 
gence. This makes it possible to assign a physical meaning to the divergence immediately. 
Also, formula (12.59) does not involve the components of F. Therefore it holds true in 
any system of coordinates. If we choose for V(t) a cube with its edges parallel to the 
xyz-coordinate axes and center at a, we can use Equation (12.59) to deduce the formula 
in (12.57) which expresses div F in terms of the components of F. This procedure is out- 
lined in Exercise 14 of Section 12.21. 

There is a formula analogous to (12.59) that is sometimes used as an alternative definition 
of the curl. It states that 


(12.61) ora =n = | | nx FdS, 
ae IVI S(t) 


where V(t) and S(t) have the same meanings as in Theorem 12.7. The surface integral 
that appears on the right has a vector-valued integrand. Such integrals can be defined in 
terms of components. The proof of (12.61) is analogous to that of Theorem 12.7. 

There is another formula involving the curl that can be deduced from (12.61) or derived 
independently. It states that 


(12.62) n- curl F(a) = lim - F 
to |S(t)| J cw) 


In this formula, S(t) is a circular disk of radius ¢t and center at a, and |S(t)| denotes its 
area. The vector m is a unit normal to S(t), and @ is the function that traces out C(t) ina 
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direction that appears counterclockwise when viewed from the tip of m. The vector field 
F is assumed to be continuously differentiable on S(t). A proof of (12.62) can be given by 
the same method we used to prove (12.59). We let g(x) =” -curl F(x) and argue as 
before, except that we use surface integrals instead of triple integrals and Stokes’ theorem 
instead of the divergence theorem. 

If F is a velocity field, the line integral over C(t) is called the circulation of F along C(t); 
the limit in (12.62) represents the circulation per unit area at the point a. Thus, a - curl F(a) 
can be regarded as a “‘circulation density’’ of F at point a, with respect to a plane perpen- 
dicular to n at a. 

When n takes the successive values i, j, and k, the dot products i- curl F, j- curl F, 
and &- curl F are the components of curl F in rectangular coordinates. When Equation 
(12.61) is taken as the starting point for the definition of curl, the formula in (12.58) for the 
rectangular components of curl F can be deduced from (12.62) in exactly this manner. 


12.21 Exercises 


1. Let S be the surface of the unit cube,O <x <1,0<y <1,0<z <1, and let m be the 
unit outer normal to S. If F(x, y, z) = x*i + yf + z?k, use the divergence theorem to evaluate 
the surface integral {Jf F-ndS. Verify the result by evaluating the surface integral directly. 


2. The sphere x? + y? ef z® = 25 is intersected by the plane z = 3. The smaller portion forms a 
solid V bounded by a closed surface Sy made up of two parts, a spherical part S, and a planar 
part S,. If the unit outer normal of V is cos «i + cos Bj + cos yk, compute the value of the 
surface integral 


i} (xz cos « + yzcos B + cos y) dS 
Ss 


if (a) S is the spherical cap S,, (b) S is the planar base S,, (c) S is the complete boundary Sp. 
Solve part (c) by use of the results of parts (a) and (b), and also by use of the divergence 
theorem. 

3. Letn = cos ai +cos Bj + cos yk be the unit outer normal to a closed surface S which bounds 
a homogeneous solid V of the type described in the divergence theorem. Assume that the 
center of mass (x, ¥, Z) and the volume |V| of V are known. Evaluate the following surface 
integrals in terms of |V| and x, j, Z. 


(a) JJ (xcos « + ycos B + zcos y)dS. 
S 

(b) JJ (xz cos « + 2yz cos B + 3z? cos y) dS. 
S 

(c) ff Q* cos « + 2xycos B — xzcos y) dS. 
S 


(d) Express ff (x? + y*)(xi + yj) -ndS in terms of the volume |V| and a moment of inertia 
of the solid. Ss 


In Exercises 4 through 10, 0f/@n and 9g/0n denote directional derivatives of scalar fields fand g 
in the direction of the unit outer normal n to a closed surface S which bounds a solid V of the type 
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described in the divergence theorem. That is, 0f/0n = Vf-nand ég/0n = Vg-n. In each of these 
exercises prove the given statement. You may assume continuity of all derivatives involved. 


+ [Ze [ffrvene. 


Na 


ON 


— 


oO 


\O 


10. 


11. 


12. 


13. 


0 
: | I a dS =0 whenever fis harmonic in V. 


[Vs 3 ds = I) /eedndy t+ [JJ Vf: Vg dx dy dz. 
Ws -st) as = : (fV'g —gV*f) dx dy dz. 


ae: fs ‘8g dS = Jase if both fand g are harmonic in V. 


[ras J] | Vf |? dx dy dz if fis harmonic in V. 
S 


7] 
V2f(a) = lim Tol hy, i iE: Tf as, where V(t) is a solid sphere of radius ¢ with center at a, 


S(t) is the surface of Vit), and |V(t)| is the volume of V(t). 


Let V be a convex region in 3-space whose boundary is a closed surface S and let m be the 
unit outer normal to S. Let F and G be two continuously differentiable vector fields such that 


curl F = curl G and div F = divG everywhere in V, 


and such that 
G:-n=F-n — everywhere on S. 


fae that F = G everywhere in V. [Hint: Let H = F — G, find a scalar field f such that 
= Vf, and use a suitable identity to prove that iS) || VF |? dx dy dz =0. From this deduce 

a H = Oin V.] 

Given a vector field G and two scalar fields f and g, each continuously differentiable on a 

convex solid V bounded by a closed surface S. Let nm denote the unit outer normal to S. Prove 

that there is at most one vector field F satisfying the following three conditions: 


culF=G and divF=g inV, F-n=f  onS. 


Let S be a smooth parametric surface with the property that each line emanating from a point 
P intersects S once at most. Let Q(S) denote the set of lines emanating from P and passing 
through S. (See Figure 12.19.) The set Q(S) is called the solid angle with vertex P subtended 
by S. Let X(a) denote the intersection of Q(S) with the surface of the sphere of radius a 
centered at P. The quotient 

area of X(a) 


a 
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FiGureE 12.19 The solid angle Q(S) with vertex P subtended by a surface S. It is 
area of X(a) 

measured by the quotient |Q(S)| = ee rer. 
is denoted by |Q(S)| and is used as a measure of the solid angle Q(S). 
(a) Prove that this quotient is equal to the surface integral 


r-n 
—z 4S, 
S 


where r is the radius vector from P to an arbitrary point of S, and r = |r||. The vector n is 
the unit normal to S directed away from P. This shows that the quotient for |Q(S)| is inde- 
pendent of the radius a. Therefore the solid angle can be measured by the area of the intersection 
of Q(¢S) and the unit sphere about P. [Hint: Apply the divergence theorem to the portion of 
QS) lying between S and X(a).] 

(b) Two planes intersect along the diameter of a sphere with center at P. The angle of inter- 
section is 6, where 0 < 8 < w. Let S denote the smaller portion of the surface of the sphere 
intercepted by the two planes. Show that |Q(S)| = 26. 

Let V(t) denote a cube of edge 2¢ and center at a, and let S(t) denote the boundary of the 
cube. Let n be the unit outer normal of S(t) and let |V(¢)| denote the volume of the cube. 
For a given vector field F that is continuously differentiable at a, assume that the following 


limit exists: 
1 
lim —— F-nds, 


S(t) 


and use this limit as the definition of the divergence, div F(a). Choose xyz-coordinate axes 
parallel to the edges of V(t) and let P, Q, and R be the components of F relative to this coordi- 
nate system. Prove that div F(a) = D,P(a) + D,Q(a) + D3R(a). [Hint: Express the surface 
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SET FUNCTIONS AND ELEMENTARY PROBABILITY 


13.1 Historical introduction 


A gambler’s dispute in 1654 led to the creation of a mathematical theory of probability by 
two famous French mathematicians, Blaise Pascal and Pierre de Fermat. Antoine 
Gombaud, Chevalier de Méré, a French nobleman with an interest in gaming and gambling 
questions, called Pascal’s attention to an apparent contradiction concerning a popular dice 
game. The game consisted in throwing a pair of dice 24 times; the problem was to decide 
whether or not to bet even money on the occurrence of at least one “‘double six’’ during 
the 24 throws. A seemingly well-established gambling rule led de Meéré to believe that 
betting on a double six in 24 throws would be profitable, but his own calculations indicated 
just the opposite. 

This problem and others posed by de Meéré led to an exchange of letters between Pascal 
and Fermat in which the fundamental principles of probability theory were formulated 
for the first time. Although a few special problems on games of chance had been solved 
by some Italian mathematicians in the 15th and 16th centuries, no general theory was 
developed before this famous correspondence. 

The Dutch scientist Christian Huygens, a teacher of Leibniz, learned of this corre- 
spondence and shortly thereafter (in 1657) published the first book on probability; entitled 
De Ratiociniis in Ludo Aleae, it was a treatise on problems associated with gambling. 
Because of the inherent appeal of games of chance, probability theory soon became popular, 
and the subject developed rapidly during the 18th century. The major contributors during 
this period were Jakob Bernoullit (1654-1705) and Abraham de Moivre (1667-1754). 

In 1812 Pierre de Laplace (1749-1827) introduced a host of new ideas and mathematical 
techniques in his book, Théorie Analytique des Probabilités. Before Laplace, probability 
theory was solely concerned with developing a mathematical analysis of games of chance. 
Laplace applied probabilistic ideas to many scientific and practical problems. The theory 
of errors, actuarial mathematics, and statistical mechanics are examples of some of the 
important applications of probability theory developed in the 19th century. 

Like so many other branches of mathematics, the development of probability theory 
has been stimulated by the variety of its applications. Conversely, each advance in the 
theory has enlarged the scope of its influence. Mathematical statistics is one important 
branch of applied probability; other applications occur in such widely different fields as 


+ Sometimes referred to as James Bernoulli. 
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genetics, psychology, economics, and engineering. Many workers have contributed to the 
theory since Laplace’s time; among the most important are Chebyshev, Markov, von 
Mises, and Kolmogorov. 

One of the difficulties in developing a mathematical theory of probability has been to 
arrive at a definition of probability that is precise enough for use in mathematics, yet 
comprehensive enough to be applicable to a wide range of phenomena. The search for a 
widely acceptable definition took nearly three centuries and was marked by much 
controversy. The matter was finally resolved in the 20th century by treating probability 
theory on an axiomatic basis. In 1933 a monograph by a Russian mathematician 
A. Kolmogorov outlined an axiomatic approach that forms the basis for the modern theory. 
(Kolmogorov’s monograph is available in English translation as Foundations of Probability 
Theory, Chelsea, New York, 1950.) Since then the ideas have been refined somewhat and 
probability theory is now part of a more general discipline known as measure theory. 

This chapter presents the basic notions of modern elementary probability theory along 
with some of its connections to measure theory. Applications are also given, primarily to 
games of chance such as coin tossing, dice, and card games. This introductory account is 
intended to demonstrate the logical structure of the subject as a deductive science and to 
give the reader a feeling for probabilistic thinking. 


13.2 Finitely additive set functions 


The area of a region, the length of a curve, or the mass of a system of particles is a 
number which measures the size or content of a set. All these measures have certain 
properties in common. Stated abstractly, they lead to a general concept called a finitely 
additive set function. Later we shall introduce probability as another example of such a 
function. To prepare the way, we discuss first some properties common to all these 
functions. 

A function f:  — R whose domain is a collection & of sets and whose function values 
are real numbers, is called a set function. If A is a set in class A, the value of the function 
at A is denoted by f(A). 


DEFINITION OF A FINITELY ADDITIVE SET FUNCTION. A Set function f: S — R is said to be 
finitely additive if 


(13.1) f(A U B) = f(A) + f(B) 


whenever A and B are disjoint sets in such that A U B is also in &. 


Area, length, and mass are all examples of finitely additive set functions. This section 
discusses further consequences of Equation (13.1). 

In the usual applications, the sets in 7 are subsets of a given set S, called a universal set. 
It is often necessary to perform the operations of union, intersection, and complementation 
on sets in . To make certain that .°7 is closed under these operations we restrict . to be 
a Boolean algebra, which is defined as follows. 
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DEFINITION OF A BOOLEAN ALGEBRA OF SETS. A nonempty class & of subsets of a given 
universal set S is called a Boolean algebra if for every A and B in L we have 


AUBEA and AES. 


Here A’ = S — A, the complement of A relative to S. 


A Boolean algebra is also closed under intersections and differences, since we have 
ANB=(A' UB’) and A—B=AQB’. 


This implies that the empty set @ belongs to YW since @ = A — A for some A in &. 
Also, the universal set S is in WY since S = @’. 

Many Boolean algebras can be formed from the subsets of a given universal set S. The 
smallest such algebra is the class Wy = {@ , S}, which consists of only two special subsets: 
@ and S. At the other extreme is the class .7,, which consists of all subsets of S. Every 
Boolean algebra consisting of subsets of S satisfies the inclusion relations WZ, c WS 
A. 

The property of finite additivity of set functions in Equation (13.1) requires A and B to 
be disjoint sets. The next theorem drops this requirement. 


THEOREM 13.1. Let f: 7 —R be a finitely additive set function defined on a Boolean 
algebra LV of sets. Then for all A and B in L we have 


(13.2) f(A UV B)=f(A) + f(B — A), 
and 
(13.3) f(A UV B) = f(A) + f(B) — f(A OB). 


Proof. Thesets A and B — A are disjoint and their union is A U B. Hence, by applying 
(13.1) to A and B — A we obtain (13.2). 

To prove (13.3) we first note that A  B’ and B are disjoint sets whose union is A U B. 
Hence by (13.1) we have 


(13.4) f(A UB) =f(A 0 B’) + f(B). 

Also, A O B’ and A OM B are disjoint sets whose union is A, so (13.1) gives us 
(13.5) (A) =f(A NB) + f(A OB). 

Subtracting (13.5) from (13.4) we obtain (13.3). 


13.3. Finitely additive measures 


The set functions which represent area, length, and mass have further properties in 
common. For example, they are all nonnegative set functions. That is, 


f(A) 2 0 


for each A in the class . under consideration. 
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DEFINITION OF A FINITELY ADDITIVE MEASURE. A nonnegative set function f:  — R that is 
finitely additive is called a finitely additive measure, or simply a measure. 


Using Theorem 13.1 we immediately obtain the following further properties of measures. 


THEOREM 13.2. Let f: % — R bea finitely additive measure defined on a Booleanalgebra ZZ. 
Then for all sets A and B in M we have 

(a) f(A UB) < f(A) +f). 

(b) f(B — A)=f(B)—-f(A) if ACB. 

(c) f(A) < f(B) if A&B. (Monotone property) 

(d) f(g) = 0. 


Proof. Part (a) follows from (13.3), and part (b) follows from (13.2). Part (c) follows 
from (b), and part (d) follows by taking A = B = @ in (b). 


EXAMPLE. The number of elements in a finite set. Let S = {a,,a,,...,a,} be a set 
consisting of n (distinct) elements, and let . denote the class of all subsets of S. For each 
A in &, let v(A) denote the number of distinct elements in A (v is the Greek letter ‘‘nu’’). 
It is easy to verify that this function is finitely additive on . In fact, if A has k elements 
and if B has m elements, then »(A) = k and »(B) = m. If A and B are disjoint it is clear 
that the union A U Bisa subset of S with k + m elements, so 


WAU B)=k+m= (A) + r(B). 


This particular set function is nonnegative, so v is a measure. 


13.4 Exercises 


1. Let denote the class of all subsets of a given universal set and let A and B be arbitrary sets 

in &W . Prove that: 

(a) A © B’ and Bare disjoint. 

(b) AUB =(A OB’) UB. (This formula expresses A U B as the union of disjoint sets.) 
(c) AQ Band A 2 B’ are disjoint. 

(d) (A OB) VU (A OB’) =A. (This formula expresses A as a union of two disjoint sets.) 

2. Exercise 1(b) shows how to express the union of two sets as the union of two disjoint sets. 
Express in a similar way the union of three sets A, U Ag U A and, more generally, of 7 sets 
A, UA, U::: UA,. Illustrate with a diagram when n = 3. 

3. A study of a set S consisting of 1000 college graduates ten years after graduation revealed 
that the “successes’’ formed a subset A of 400 members, the Caltech graduates formed a subset 
B of 300 members, and the intersection A © B consisted of 200 members. 

(a) For each of the following properties, use set notation to describe, in terms of unions and 
intersections of A, B, and their complements 4’ and B’ relative to S, the subsets consisting of 
those persons in S that have the property: 

(1) Neither a “success” nor a Caltech graduate. 

(i) A “success” but not a Caltech graduate. 

(ili) A “‘success”’ or a Caltech graduate, or both. 


4. 
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(iv) Either a “‘success”’ or a Caltech graduate, but not both. 

(v) Belongs to not more than one of A or B. 
(b) Determine the exact number of individuals in each of the five subsets of part (a). 
Let f be a finitely additive set function defined on a class of sets. Let A,,...,A, be nsets 
in Y such that A; A A, = @ ifi #/7. (Such acollection is called a disjoint collection of sets.) 


m 
If the union L) A, is in class YW for all m <n, use induction to prove that 
k=1 


In Exercises 5, 6, and 7, S denotes a finite set consisting of n distinct elements, say S = 


{Qisi@ey oes oats 


5. 


6. 


iE 
8. 


10. 


Let A, = {a,}, the subset consisting of a, alone. Show that theclass #, = {@, A,, Aj, Sis 
the smallest Boolean algebra containing A,. 

Let A, = {a,}, Ay = {a.}. Describe, ina manner similar to that used in Exercise 5, the smallest 
Boolean algebra #, containing both A, and A,. 

Do the same as in Exercise 6 for the subsets A, = {a,}, Ag = {ap}, and Az = {ag}. 

If @, denotes the smallest Boolean algebra which contains the k subsets 4, = {a,}, Ap = 
{ay},..., Ay = {a;,}, show that F, contains 2*+! subsets of Sifk < nand 2” subsets ifk =n. 


. Let fbe a finitely additive set function defined on the Boolean algebra of all subsets of a given 


universal set S. Suppose it is known that 


f(A 0 B) = f(A) f(B) 


for two particular subsets A and B of S. If f(S) = 2, prove that 
f(A VB) = f(A) +f) - fA. 


If A and Bare sets, their symmetric difference A A Bis the set defined by theequation A A B = 
(A — B) U(B — A). Prove each of the following properties of the symmetric difference. 

(a) AAB=BAA. 

(b) AAA=Q. 

(C) AABC(AAC)YU(C AB). 

(d) A A Bis disjoint from each of A and B. 

(ec) (A AB) AC=A A(BAC). 

(f) If fis a finitely additive set function defined on the Boolean algebra .° of all subsets of a 
given set S, then for all A and Bin W we have f(A A B) = f(A) + f(B) — 2f(4 OB). 


13.5 The definition of probability for finite sample spaces 


In the language of set functions, probability is a specific kind of measure (to be denoted 


here by P) defined on a specific Boolean algebra & of sets. The elements of #& are subsets 
of a universal set S. In probability theory the universal set S is called a sample space. We 


discuss the definition of probability first for finite sample spaces and later for infinite sample 


spaces. 


DEFINITION OF PROBABILITY FOR FINITE SAMPLE SPACES. Let & denote a Boolean algebra 


whose elements are subsets of a given finite set S. A set function P defined on & is called a 
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probability measure if it satisfies the following three conditions: 
(a) P is finitely additive. 
(b) P is nonnegative. 
(c) P(S) = 1. 


In other words, for finite sample spaces probability is simply a measure which assigns the 
value | to the whole space. 

It is important to realize that a complete description of a probability measure requires 
three things to be specified: the sample space S, the Boolean algebra 4 formed from certain 
subsets of S, and the set function P. The triple (S, #, P) is often called a probability space. 
In most of the elementary applications the Boolean algebra @ is taken to be the collection 
of all subsets of S. 


EXAMPLE. An illustration of applied probability theory is found in the experiment of 
tossing a coin once. For a sample space S we take the set of all conceivable outcomes of 
the experiment. In this case, each outcome is either “heads”’ or “‘tails,’’ which we label by the 
symbols A and t. Thus, the sample space S is {h, t}, the set consisting of h and ¢. For the 
Boolean algebra we take the collection of all subsets of S; there are four, @, S, H, and 7, 
where H = {h} and T = {t}. Next, we assign probabilities to each of these subsets. For 
the subsets @ and S we have no choice in the assignment of values. Property (c) requires 
that P(S) = 1, and, since P is a nonnegative measure, P(@) = 0. However, there is some 
freedom in assigning probabilities to the other two subsets, H and 7. Since H and T are 
disjoint sets whose union is S, the additive property requires that 


P(H) + P(T) = P(S) = 1. 


We are free to assign any nonnegative values whatever to P(#) and P(7) so long as their 
sum is 1. If we feel that the coin is unbiased so that there is no a priori reason to prefer 
heads or tails, it seems natural to assign the values 


P(H) = P(T) = }. 


If, however, the coin is “loaded,” we may wish to assign different values to these two 
probabilities. For example, the values P(H) = 3 and P(T) = § are just as acceptable as 
P(H) = P(T) = 3%. In fact, for any real p in the interval O0<p<1 we may define 
P(H) = p and P(T) = | — p, and the resulting function P will satisfy all the conditions 
for a probability measure. 

For a given coin, there is no mathematical way to determine what the probability p 
“really” is. If we choose p = 3 we can deduce logical consequences on the assumption that 
the coin is fair or unbiased. The theory for unbiased coins can then be used to test the 
fairness of an actual coin by performing a large number of experiments with the coin and 
comparing the results with the predictions based on the theory. The testing of agreement 
between theory and empirical evidence belongs to that branch of applied probability known 
as Statistical inference, and will not be discussed in this book. 


The foregoing example is a typical application of the concepts of probability theory. 
Probability questions often arise in situations referred to as “experiments.” We shall not 
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attempt to define an experiment; instead, we shall merely mention some familiar examples: 
tossing one or more coins, rolling a pair of dice, dealing a bridge hand, drawing a ball from 
an urn, counting the number of female students at the California Institute of Technology, 
selecting a number from a telephone directory, recording the radiation count of a Geiger 
counter. 

To discuss probability questions that arise in connection with such experiments, our 
first task is to construct a sample space S that can be used to represent all conceivable 
outcomes of the experiment, as we did for coin tossing. Each element of S should represent 
an outcome of the experiment and each outcome should correspond to one and only one 
element of S. Next, we choose a Boolean algebra # of subsets of S (usually all subsets of 
S) and then define a probability measure P on &. The choice of the set S, the choice of Z, 
and the choice of P will depend on the information known about the details of the experi- 
ment and on the questions we wish to answer. The purpose of probability theory is not to 
discuss whether the probability space (S, @, P) has been properly chosen. This motivation 
belongs to the science or gambling game from which the experiment emanates, and only 
experience can suggest whether or not the choices were well made. Probability theory is the 
study of logical consequences that can be derived once the probability space is given. Making 
a good choice of the probability space is, strictly speaking, not probability theory — it is 
not even mathematics; instead, it is part of the art of applying probability theory to the 
real world. We shall elaborate further on these remarks as we deal with specific examples 
in the later sections. 

If S = {a,, a,,...,a,}, and if @ consists of all subsets of S, the probability function P 
is completely determined if we know its values on the one-element subsets, or singletons: 


P({a,}), P({a2}),... , P({a,})- 


In fact, every subset A of S is a disjoint union of singletons, and P(A) is determined by the 
additive property. For example, when 


A = {a} U {a} U-ss U tay}, 
the additive property requires that 
k 
P(A) = > P({a;}). 
7=1 


To simplify the notation and the terminology, we write P(a;) instead of P({a,}). This 
number is also called the probability of the point a,;. Therefore, the assignment of the point 
probabilities P(x) for each element x in a finite set S amounts to a complete description of 
the probability function P. 


13.6 Special terminology peculiar to probability theory 


In discussions involving probability, one often sees phrases from everyday language 
such as “two events are equally likely,’ ‘‘an event is impossible,”’ or “‘an event is certain 
to occur.”” Expressions of this sort have intuitive appeal and it is both pleasant and helpful 
to be able to employ such colorful language in mathematical discussions. Before we can do 
so, however, it is necessary to explain the meaning of this language in terms of the funda- 
mental concepts of our theory. 

Because of the way probability is used in practice, it is convenient to imagine that each 
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probability space (S, 4, P) is associated with a real or conceptual experiment. The 
universal set S can then be thought of as the collection of all conceivable outcomes of the 
experiment, as in the example of coin tossing discussed in the foregoing section. Each 
element of S is called an outcome or a sample and the subsets of S that occur in the Boolean 
algebra B are called events. The reasons for this terminology will become more apparent 
when we treat some examples. 

Assume we have a probability space (S, #, P) associated with an experiment. Let A 
be an event, and suppose the experiment is performed and that its outcome is x. (In other 
words, let x be a point of S.) This outcome x may or may not belong to the set A. If 
it does, we say that the event A has occurred. Otherwise, we say that the event A has not 
occurred, in which case x € A’, so the complementary event A’ has occurred. An event A 
is called impossible if A = @ , because in this case no outcome of the experiment can be 
an element of A. The event A 1s said to be certain if A = S, because then every outcome 
is automatically an element of A. 

Each event A has a probability P(A) assigned to it by the probability function P. [The 
actual value of P(A) or the manner in which P(A) is assigned does not concern us at 
present.] The number P(A) is also called the probability that an outcome of the experiment 
is one of the elements of A. We also say that P(A) is the probability that the event A occurs 
when the experiment is performed. 

The impossible event @ must be assigned probability zero because P is a finitely additive 
measure. However, there may be events with probability zero that are not impossible. 
In other words, some of the nonempty subsets of S may be assigned probability zero. The 
certain event S must be assigned probability 1 by the very definition of probability, but 
there may be other subsets as well that are assigned probability 1. In Example | of Section 
13.8 there are nonempty subsets with probability zero and proper subsets of S that have 
probability 1. 

Two events A and B are said to be equally likely if P(A) = P(B). The event A 1s called 
more likely than B if P(A) > P(B), and at least as likely as B if P(A) > P(B). Table 13.1 
provides a glossary of further everyday language that is often used in probability discus- 
sions. The letters A and B represent events, and x represents an outcome of an experiment 
associated with the sample space S. Each entry in the left-hand column is a statement 
about the events A and B, and the corresponding entry in the right-hand column defines the 
statement in terms of set theory. 


TABLE 13.1. Glossary of Probability Terms 


Statement Meaning in set theory 
At least one of A or B occurs xE€AUB 
Both events A and B occur xEAOB 
Neither A nor B occurs xEA OB 
A occurs and B does not occur xE€ AB 
Exactly one of A or B occurs x€(A OB’) U(A' OB) 
Not more than one of A or B occurs xE€(A BY 
If A occurs, so does B (A implies B) ACB 
A and B are mutually exclusive ANB=2 
Event A or event B AUB 


Event A and event B ANB 
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13.7 Exercises 


Let S be a given sample space and let A, B, and C denote arbitrary events (that is, subsets of S 
in the corresponding Boolean algebra #). Each of the statements in Exercises 1 through 12 is 
described verbally in terms of A, B, C. Express these statements in terms of unions and intersections 
of A, B, C and their complements. 


1. If A occurs, then B does not occur. 7. At least two of A, B, C occur. 
2. None of the events A, B, C occurs. 8. Exactly two occur. 

3. Only A occurs. 9. Not more than two occur. 

4. At least one of A, B, C occurs. 10. A and C occur but not B. 

5. Exactly one of A, B, C occurs. 11. All three events occur. 

6. Not more than one occurs. 12. Not more than three occur. 


13. Let A denote the event of throwing an odd total with two dice, and let B denote the event of 
throwing at least one 6. Give a verbal description of each of the following events: 


(a) AUB, (d) A’ OB, 
(b)A OB, (ce) A’ OB’, 
(c)A OB’, (f) A’ UB. 


14. Let A and B denote events. Show that 
P(A OB) < P(A) < P(A UB) < P(A) + P(B). 


15. Let A and B denote events and let a = P(A), b = P(B),c = P(A % B). Compute, in terms of 
a, b, and c, the probabilities of the following events: 


(a) A’, (d) A’ UB’, 
(b) B’, (e) A’ UB, 
(c) AUB, (f) ANB’. 


16. Given three events A, B, C. Prove that 


P(A UB UC) = P(A) + P(B) + P(C) — P(A OB) —- P(ANC)—-P(BOAC)+P(ANBOC). 


13.8 Worked examples 


We shall illustrate how some of the concepts of the foregoing sections may be used to 
answer specific questions involving probabilities. 


EXAMPLE 1. What is the probability that at least one “head” will occur in two throws 
of a coin? 


First Solution. The experiment in this case consists of tossing a coin twice; the set S 
of all possible outcomes may be denoted as follows: 


S = {hh, ht, th, tt}. 


If we feel that these outcomes are equally likely, we assign the point probabilities P(x) = 4 
for each x in S. The event “at least one head occurs’’ may be described by the subset 


A = hh, ht, thy. 
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The probability of this event is the sum of the point probabilities of its elements. Hence, 
PA = 2g a = a 


Second Solution. Suppose we use the same sample space but assign the point proba- 
bilities as follows:T 
Pihh)=1, = Pht) = P(th) = P(tt) = 0. 


Then the probability of the event ‘“‘at least one head occurs’”’ is 
P(hh) + P(At) + P(th) =1+0+90=1. 


The fact that we arrived at a different answer from that in the first solution should not 
alarm the reader. We began with a different set of premises. Psychological considerations 
might lead us to believe that the assignment of probabilities in the first solution is the more 
natural one. Indeed, most people would agree that this is so if the coin is unbiased. How- 
ever, if the coin happens to be loaded so that heads always turns up, the assignment of 
probabilities in the second solution is more natural. 


The foregoing example shows that we cannot expect a unique answer to the question 
asked. To answer such a question properly we must specify the choice of sample space 
and the assignment of point probabilities. Once the sample space and the point proba- 
bilities are known only one probability for a given event can be logically deduced. Different 
choices of the sample space or point probabilities may lead to different “‘correct”” answers 
to the same question. 

Sometimes the assignment of probabilities to the individual outcomes of an experiment 
is dictated by the language used to describe the experiment. For example, when an object 
is chosen “‘at random’ from a finite set of n elements, this is intended to mean that each 
outcome is equally likely and should be assigned point probability 1/n. Similarly, when we 
toss a coin or roll a die, if we have no a priori reason to feel that the coin or die is loaded, 
we assume that all outcomes are equally likely. This agreement will be adopted in all the 
exercises of this chapter. 


EXAMPLE 2. If one card is drawn at random from each of two decks, what is the proba- 
bility that at least one is the ace of hearts? 


Solution. The experiment consists in drawing two cards, a and b, one from each deck. 
Suppose we denote a typical outcome as an ordered pair (a, b). The number of possible 
outcomes, that is, the total number of distinct pairs (a, b) in the sample space S is 52?. 
We assign the probability 1/52? to each such pair. The event in which we are interested 
is the set A of pairs (a, b), where either a or b is the ace of hearts. There are 52 + 51 
elements in A. Hence, under these assumptions we deduce that 


s2+51 1 1 
P(A) = ——- =— - =. 
@ 52? 26 «528 


+ Note that for this assignment of probabilities there are nonempty subsets of S with probability zero and 
proper subsets with probability 1. 
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EXAMPLE 3. If two cards are drawn at random from one deck, what is the probability 
that one of them is the ace of hearts? 


Solution. As in Example 2 we use ordered pairs (a, b) as elements of the sample space. 
In this case the sample space has 52-51 elements and the event A under consideration 
has 51 + 51 elements. If we assign the point probabilities 1/(52 +51) to each outcome, 
we obtain 

2°51 | 


( — ————— — 


«52°51 26. 
EXAMPLE 4. What 1s the probability of throwing 6 or less with three dice? 


Solution. We denote each outcome of the experiment as a triple of integers (a, b, c) 
where a, b, and c may take any values from 1 to 6. Therefore the sample space consists 
of 6 elements and we assign the probability 1/6? to each outcome. The event A in question 
is the set of all triples satisfying the inequality 3< a+b+c<6. If A, denotes the 
set of (a, b, c) for which a + b + c =n, we have 


A= 4A,UA,UA; U A,. 


Direct enumeration shows that the sets A,, with n = 3, 4, 5, and 6 contain 1, 3, 6, and 10 
elements, respectively. For example, the set A, is given by 
A, = {(1, 2, 3), (1, 3, 2), C1, 1, 4), Cl, 4, LD), (2, 1, 3), 

(2, 3, 1), (2, 2, 2), (3, 1, 2), (3, 2, 1), (4, 1, D}. 


Therefore A has 20 elements and 


EXAMPLE 5. A die is thrown once. What 1s the probability that the number of points 
is either even or a multiple of 3? 


Solution. We choose the sample space S = {1, 2, 3, 4, 5, 6}, consisting of six elements, 
to each of which we assign the probability §. The event “even’’ is the set A = {2, 4, 6}, 
the event “a multiple of 3” is B = {3,6}. We are interested in their union, which is the 
set A U B = {2, 3, 4, 6}. Since this set contains four elements we have P(A U B) = 4/6. 

This example can be solved in another way, using the formula 


P(A U B) = P(A) + P(B)— PANB)=6+%-6- 


13.9 Exercises 


1. Let S bea finite sample space consisting of n elements. Suppose we assign equal probabilities to 
each of the points in S. Let A bea subset of S consisting of k elements. Prove that P(A) = k/n. 


For each of Exercises 2 through 8, describe your choice of sample space and state how you are 
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assigning the point probabilities. In the questions associated with card games, assume all cards 
have the same probability of being dealt. 


2. Five counterfeit coins are mixed with nine authentic coins. 

(a) A coin is selected at random. Compute the probability that a counterfeit coin is selected. 
If two coins are selected, compute the probability that: 

(b) one is good and one is counterfeit. 

(c) both are counterfeit. 

(d) both are good. 

3. Compute the probabilities of each of the events described in Exercise 13 of Section 13.7. Assign 
equal probabilities to each of the 36 elements of the sample space. 

4. What is the probability of throwing at least one of 7, 11, or 12 with two dice? 

5. A poker hand contains four hearts and one spade. The spade is discarded and one card is 
drawn from the remainder of the deck. Compute the probability of filling the flush—that is, of 
drawing a fifth heart. 

6. In poker, a straight is a five-card sequence, not necessarily all of the same suit. If a poker hand 
contains four cards in sequence (but not A234 or JQKA) and one extra card not in the sequence, 
compute the probability of filling the straight. (The extra card is discarded and a new card is 
drawn from the remainder of the deck.) 

7. A poker hand has four cards out of a five-card sequence with a gap in the middle (such as 
5689), and one extra card not in the sequence. The extra card is discarded and a new one is 
drawn from the remainder of the deck. Compute the probability of filling the ‘inside straight.” 

8. Anurn contains A white stones and B black stones. A second urn contains C white stones and 
D black stones. One stone is drawn at random from the first urn and transferred to the second 
urn. Then a stone is drawn at random from the second urn. Calculate the probability of each 
of the following events. 

(a) The first stone is white. 
(b) The first stone is black. 
(c) The second stone is white, given that the transferred stone was white. 
(d) The second stone is white, given that the transferred stone was black. 
9. Two stones are drawn with replacement from an urn containing four red stones and two white 
stones. Calculate the probability of each of the following events. 
(a) Both stones are white. 
(b) Both stones are red. 
(c) Both stones are the same color. 
(d) At least one stone is red. 
10. Let P,, denote the probability that exactly n of the events A and B will occur, where n takes the 
values 0, 1, 2. Express each of Py, P,, P, in terms of P(A), P(B), and P(A 2 B). 


Odds. Some gambling games are described in terms of “odds’’ rather than in terms of 
probabilities. For example, if we roll a fair die, the probability of the event “rolling a three” is 3. 
Since there are six possible outcomes, one of which is favorable to the event “rolling a three” and 
five of which are unfavorable, this is often described by saying that the odds in favor of the event 
are 1 to 5, or the odds against it are 5 to 1. In this case the odds are related to the probability by 
the equation 

1 1 


rs ee 


In general, if A is an event with probability P(A) and if a and b are two real numbers such that 


(13.6) P(A) = — 
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we say the odds in favor of A are a to b, or the odds against Aare btoa. Since 1 — a/(a + b) = 
b/(a + b), the odds against A are the same as the odds in favor of the complementary event A’. 
The following exercises are devoted to further properties of odds and their relation to probabilities. 
11. If P(A) = 1, show that (13.6) can be satisfied only when 6 = 0 anda #0. If P(A) #1, 
show that there are infinitely many choices of a and 5b satisfying (13.6) but that all have the 
same ratio a/b. 
12. Compute the odds in favor of each of the events described in Exercise 2. 
13. Given two events A and B. If the odds against A are 2 to 1 and those in favor of A U B are 
3 to 1, show that 
ts < P(B) <2. 


Give an example in which P(B) = 35 and one in which P(B) = ?. 


13.10 Some basic principles of combinatorial analysis 


Many problems in probability theory and in other branches of mathematics can be 
reduced to problems on counting the number of elements in a finite set. Systematic methods 
for studying such problems form part of a mathematical discipline known as combinatorial 
analysis. In this section we digress briefly to discuss some basic ideas in combinatorial 
analysis that are useful in analyzing some of the more complicated problems of probability 
theory. 

If all the elements of a finite set are displayed before us, there is usually no difficulty in 
counting their total number. More often than not, however, a set is described in a way 
that makes it impossible or undesirable to display all its elements. For example, we might 
ask for the total number of distinct bridge hands that can be dealt. Each player is dealt 
13 cards from a 52-card deck. The number of possible distinct hands is the same as the 
number of different subsets of 13 elements that can be formed from a set of 52 elements. 
Since this number exceeds 635 billion, a direct enumeration of all the possibilities is clearly 
not the best way to attack this problem; however, it can readily be solved by combinatorial 
analysis. 

This problem is a special case of the more general problem of counting the number of 
distinct subsets of k elements that may be formed from a set of n elements,f where n > k. 
Let us denote this number by f(”, k). It has long been known that 


(13.7) fin, b= (7), 
where, as usual (7) denotes the binomial coefficient, 


Serer 


In the problem of bridge hands we have /(52, 13) = (73) = 635,013,559,600 different 
hands that a player can be dealt. 


t+ When we say that a set has n elements, we mean that it has n distinct elements. Such a set is sometimes 
called an n-element set. 
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There are many methods known for proving (13.7). A straightforward approach is to 
form each subset of k elements by choosing the elements one at a time. There are n 
possibilities for the first choice, n — 1 possibilities for the second choice, and n — (k — 1) 
possibilities for the kth choice. If we make all possible choices in this manner we obtain 
a total of 

n! 
n(n — 1)°*-(n—k + 1) = —— 
(n — 1) +++ os 
subsets of k elements. Of course, these subsets are not all distinct. For example, if k = 3 
the six subsets 
{a, b, c}, {b, c, a}, {c, a, b}, {a, c, b}, {c, b, a}, {b, a, c} 


arc all equal. In general, this method of enumeration counts each k-element subset exactly 
k! times.t Therefore we must divide the number n!/(n — k)! by k! to obtain f(n, k). This 
gives us f(n, k) = (Z), as asserted. 

This line of argument is more or less typical of the combinatorial analysis required in 
the later sections. Hence it seems worthwhile to digress briefly to discuss the fundamental 
principles on which this analysis is based. 

We often wish to count the number of elements in the Cartesian product of n finite sets 
A,,..., A,. The Cartesian product is denoted by the symbol A, x --- X A, and Is 
defined by the equation 


Ay X +++ X An = ((Q,- + An) | 1 © At, «- «5 Ay © An}. 


That is, the Cartesian product consists of the set of all ordered n-tuples (a,,..., a,) where 
the kth component of the n-tuple comes from the kth set A,. 

An example with n = 2 is shown in Figure 13.1. Here A, = {1, 2, 4, 5} and A, = {1, 3}. 
There are 4 elements in A,, and 2 elements in Ag, giving a total of 8 elements in the Cartesian 
product A, x A,. More generally, if A, consists of k, elements and A, consists of k, 
elements, then A, x A, consists of kik, elements. By induction on a, it follows that if A, 
consists of k, elements, then the Cartesian product A, xX --- xX A, consists of k,---k, 
elements. 

To express this result in the language of set functions, let A denote the class of all 
finite sets and let v be the set function defined on F as follows: If A € ¥ , v(A) is the number 
of distinct elements in A. (For the empty set we define »(@) = 0.) Then it is easy to verify 
that » is a finitely additive set function, so we may write 


(13.8) »(U s,) = > (S;) 

t=1 i=1 
if {S,, S,,..., S,} is a disjoint collection of finite sets (that is, if S; A S; = @ whenever 
i #j). The number of elements in a Cartesian product may be expressed in terms of » 


as follows: 
v(Ay X Ag X +++ X A,) = 0(A1)(AQ) ++ 9(A,). 


+ The reason for this will become clear in Example 3 on p. 484, where we give a more detailed derivation 
of (13.7). 


Some basic principles of combinatorial analysis 483 


ay 


A, = {l, 2, 4, 5} 


FiGurE 13.1 An example illustrating the Cartesian of two sets. The plotted points 
represent A, X Ag. 


A similar formula tells us how to count the number of elements in any set 7 of n-tuples 
if we know the number of possible choices for each of the successive components. For 
example, suppose there are k, possible choices for the first component x,. Let k, be the 
number of possible choices for the second component x,, once x, is known. Similarly, 
let k, be the number of possible choices for the rth component x,, once x1, X2,..., X;-1 
are known. Then the number of n-tuples that can be formed with these choices is 


W(T) = kyk,°+ + k,. 


This formula is often referred to as the principle of sequential counting. It can be proved 
by induction onz. In many applications the set of choices for x, may not be easy to describe 
since it may not be determined until after the choices of the earlier components have been 
made. (This was the case when we used the principle to count bridge hands.) Fortunately, 
to apply the principle of sequential counting we do not need to know the actual set of 
choices for x, but only the number of possible choices for x,. 

The additive property in formula (13.8) and the principle of sequential counting provide 
the key to the solution of many counting problems. The following examples show how they 
can be applied. 


EXAMPLE |. Sampling with replacement. Given a set S consisting of n elements. If 
k > 1, how many ordered k-tuples can be formed if each component may be an arbitrary 
element of S? 


Note: It may be helpful to think of S as an urn containing n balls labeled 1, 2,..., 2. 
We select a ball and record its label as the first component of our k-tuple. Replacing 
the ball in the urn, we again select a ball and use its label as the second component, 
and so on, until we have made k selections. Since we replace each ball after it is drawn, 
the same label may appear in different components of our k-tuple. 
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Solution. Each k-tuple is an element of the Cartesian product 
T=S,X°°'X &, 


where each S; = S. Conversely, each element of T is one of the k-tuples in question. 
Hence the number of k-tuples formed in this way is 


v(T) = v(S,)° ++ v(S,) = n*. 


EXAMPLE 2. Sampling without replacement. Given a set S consisting of 2 elements. If 
k <n, how many ordered k-tuples can be formed if the components are chosen from S 
without replacement, that is to say, if no element of S may be used twice in any given 
k-tuple? 


Solution. Consider any k-tuple (x,, x2, ..., X,) formed from the elements of S without 
replacement. For the first component x, there are n choices (the n elements of S'). When 
x, is chosen there remain n — 1 ways to choose x,. With x, chosen, there remain n — 2 
ways to choose x3, and so on, there being n — k + 1 ways to choose x,. Therefore, by 
the principle of sequential counting, the total number of k-tuples so formed is 


n! 
n(n — 1)(n — 2):--(n —k + 1) = ———. 
(n — 1m — 2)+> + es 
In particular, when k = n this result tells us that m! distinct n-tuples may be formed from 
a given set S of n elements, with no two components of any n-tuple being equal. 


EXAMPLE 3. The number of k-element subsets of an n-element set. If k <n, how many 
distinct subsets of k elements can be formed from a set S consisting of n elements? 


Solution. Let r denote the number of subsets in question and let us denote these subsets 


by 
Ais, Ais nigh 


These sets are distinct but need not be disjoint. We shall compute r in terms of n and k 
by an indirect method. For this purpose, let B; denote the collection of k-tuples that can 
be formed by choosing the components from the elements of A; without replacement. 
The sets B,, B,,..., B, are disjoint. Moreover, if we apply the result of Example 2 with 
n = k we have 
v(B,) = k! foreachi=1,2,...,r. 

Now let 

T=B,UBU>*: UB... 
Then 7 consists of all k-tuples that can be formed by choosing the components from S$ 
without replacement. From Example 2 we have 


v(T) = n'/(n — k)! 
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and by additivity we also have 
W(T) = > o(B) =k! r. 


e 


i=1 


Equating the two expressions for »(7) we obtain 


a 7 — ‘! (;) : 


This proves formula (13.7) stated earlier in this section. 


If we use the result of Example 3 to count the total number of subsets of a set S consisting 
of n elements, we obtain 


k=0 


Since this sum is also obtained when we expand (1 + 1)” by the binomial theorem, the 
number of subsets of S is 2”. 


13.11 Exercises 


1. Let A = {1, 2,3}. Display in roster notation the set of ordered pairs (a, b) obtained by 
choosing the first component from A and the second component from the remaining elements 
of A. Can this set of pairs be expressed as a Cartesian product? 

2. A two-card hand can be dealt from a deck of 52 cards in 52:51 = 2652 ways. Determine the 
number of distinct hands, and explain your reasoning. 

3. A senate committee consisting of six Democrats and four Republicans is to choose a chairman 
and a vice-chairman. In how many ways can this pair of officers be chosen if the chairman 
must be a Democrat? 

4. An experiment consists of tossing a coin twice and then rolling a die. Display each outcome 
of this experiment as an ordered triple (a, b, c), where each of a and 5 is either H (heads) 
or T (tails) and c is the number of points on the upturned face of the die. For example, 
(H, H, 3) means that heads came up on both tosses and 3 appeared on the die. Express the 
set of all possible outcomes as a Cartesian product and determine the number of possible 
outcomes. 

5. In how many ways can a bridge deck of 52 cards be dealt into four hands, each containing 13 
cards? Explain your reasoning. 

6. Two dice, one red and one white, are tossed. Represent the outcome as an ordered pair (a, b), 
where a denotes the number of points on the red die, b the number on the white die. How 
many ordered pairs (a, b) are possible? How many are there for which the suma + bis: 

(a) even? (b) divisible by 3? (c) either even or divisible by 3? 

7. A poker hand contains five cards dealt from a deck of 52. How many distinct poker hands 

can be dealt containing: 

(a) two pairs (for example, 2 kings, 2 aces, and a 3)? 

(b) a flush (five cards in a given suit)? 

(c) a straight flush (any five in sequence in a given suit, but not including ten, jack, queen, 
king, ace)? 

(d) a royal flush (ten, jack, queen, king, ace in a single suit)? 
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8. Refer to Exercise 7. Compute the probability for a poker hand to be: 
(a) a flush. 
(b) a straight flush. 
(c) a royal flush. 
9. How many committees of 50 senators can be formed that contain: 
(a) exactly one senator from Alaska? 
(b) both senators from Alaska? 
10. A committee of 50 senators is chosen at random. Compute the probability that both senators 
from Alaska are included. 
11. A code group consists of four symbols in a row, each symbol being either a dot or a dash. 
How many distinct code groups can be formed? 
12. How many &-letter words can be formed with an alphabet containing v letters? 
13. Show that: 


HL) CoG) eG) Geom 
© (e(h +07 -G) 


14. Suppose a set of ordered pairs (a, b) is constructed by choosing the first component from a 
set of k elements, say from the set {a,,...,a,}, and the second component from a set of m 
elements, say from {b,,..., b,,}. There are a total of m pairs with first component a, , namely, 
(a), 5,),..., (@,, bm). Similarly, there are m pairs (a;, b,),... , (@;, bm) with first component 
a;. Therefore the total number of ordered pairs (a, b) is m + m +--+ +m (k summands). 


This sum equals km, which proves the principle of sequential counting for sets of ordered 
pairs. Use induction to prove the principle for sets of ordered n-tuples. 


13.12 Conditional probability 


An unbiased die is thrown and the result is known to be an even number. What is the 
probability that this number is divisible by 3? What is the probability that a child is color 
blind, given that it is a boy? These questions can be put in the following form: Let A and 
B be events of a sample space S. What is the probability that A occurs, given that B has 
occurred? This is not necessarily the same as asking for the probability of the event 
AB. In fact, when A = B the question becomes: If A occurs, what is the probability 
that A occurs? The answer in this case should be 1 and this may or may not be the proba- 
bility of A 1 B. To see how to treat such problems in general, we turn to the question 
pertaining to rolling a die. 

When we ask probability questions about rolling an unbiased die, we ordinarily use 
for the sample space the set S = {1, 2, 3, 4, 5, 6} and assign point probability ¢ to each 
element of S. The event “divisible by 3” is the subset A = {3, 6} and the event “‘even”’ is 
the subset B = {2, 4, 6}. We want the probability that an element is in A, given that it is 
in B. Since we are concerned only with outcomes in which the number is even, we disregard 
the outcomes 1, 3, 5 and use, instead of S, the set B = {2, 4, 6} as our sample space. The 
event in which we are now interested is simply the singleton {6}, this being the only outcome 
of the new sample space that is divisible by 3. If all outcomes of B are considered equally 
likely, we must assign probability $ to each of them; hence, in particular, the probability of 
{6} is also 3. 

Note that we solved the foregoing problem by employing a very elementary idea. We 
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simply changed the sample space from S to B and provided a new assignment of probabilities. 
This example suggests a way to proceed in general. 

Let (S, &, P) be a given probability space. Suppose A and B are events and consider 
the question: “What is the probability that A occurs, given that B has occurred?” 
As in the example just treated, we can change the sample space from S to B and provide a 
new assignment of probabilities. We are at liberty to do this in any manner consistent with 
the definition of probability measures. For B itself we have no choice except to assign the 
probability 1. Since we are interested in those elements of A which lie in the new sample 
space B, the problem before us is to compute the probability of the event A \ B according 
to the new assignment of probabilities. That is, if P’ denotes the probability function 
associated with the new sample space B, then we must compute P’(A A B). 

We shall show now that if P(B) ¥ 0 we can always define a probability function P’ 
and a Boolean algebra # of subsets of B such that (B, &', P’) is a probability space. For 
the Boolean algebra @ we take the collection of all sets T © B where T is in the original 
Boolean algebra &. It is easy to verify that @’, so defined, is indeed a Boolean algebra. 
One way to define a probability function P’ on & is simply to divide each of the old 
probabilities by P(B). That is, if Ce B we let 


PQ =f. 

P(B) 
(This is where the assumption P(B) ¥ 0 comes in.) We are only changing the scale, with 
all probabilities magnified by the factor 1/P(B). It is easy to check that this definition of 
P" gives us a bona fide probability measure. It is obviously nonnegative and it assigns 
probability 1 to B. The additive property follows at once from the corresponding additive 
property for P. 

Since each C in @" is of the form A  B, where A is an event in the original sample 
space S, we can rewrite the definition of P’ as follows: 


P(A OB) 


P(A OB)= P(B) 


This discussion suggests that the quotient P(A M B)/P(B) provides a reasonable measure 
of the probability that A occurs, given that B occurs. The following definition is made 
with this motivation in mind: 


DEFINITION OF CONDITIONAL PROBABILITY. Let (S, 4, P) be a probability space and let 
B be an event such that P(B) #0. The conditional probability that an event A will occur, 
given that B has occurred, is denoted by the symbol P(A | B) (read: ‘‘the probability of A, 
given B’’) and is defined by the equation 
P(A OB) 


P(A |B) = PB) 


The conditional probability P(A | B) is not defined if P(B) = 0. 


The following examples illustrate the use of the concept of conditional probability. 
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EXAMPLE |. Let us consider once more the problem raised earlier in this section: A die 
is thrown and the result is known to be an even number. What is the probability that this 
number is divisible by 3? As a problem in conditional probabilities, we may take for the 
sample space the set S = {1, 2, 3, 4, 5, 6} and assign point probabilities § to each element 
of S. The event “‘even”’ is the set B = {2, 4, 6} and the event “divisible by 3”’ is the set 
A = {3,6}. Therefore we have 


This agrees, of course, with the earlier solution in which we used B as the sample space 
and assigned probability 4 to each element of B. 


EXAMPLE 2. This is an example once used by the Caltech Biology Department to warn 
against the fallacy of superficial statistics. To “‘prove”’ statistically that the population of 
the U.S. contains more boys than girls, each student was asked to list the number of boys 
and girls in his family. Invariably, the total number of boys exceeded the total number of 
girls. The statistics in this case were biased because all undergraduates at Caltech were males. 
Therefore, the question considered here is not concerned with the probability that a child 
is a boy; rather, it is concerned with the conditional probability that a child is a boy, given 
that he comes from a family with at least one boy. 

To compute the probabilities in an example of this type consider a sample of 4n families, 
each with two children. Assume that n families have 2 boys, 2n families one boy and one 
girl, and n families 2 girls. Let the sample space S' be the set of all 8n children in these 
families and assign the point probability P(x) = 1/(8n) to each x in S. Let A denote the 
event “the child is a boy” and B the event “‘the child comes from a family with at least one 
boy.’”’ The probability P(A) is obviously 3. Similarly, P(B) = ? since 3n of the 4n families 
have at least one boy. Therefore the probability that a child is a boy, given that he comes 
from a family with at least one boy, is the conditional probability 
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13.13 Independence 


An important idea related to conditional probability is the concept of independence of 
events, which may be defined as follows: 


DEFINITION OF INDEPENDENCE. Tivo events A and B are called independent (or stochasti- 
cally independent) if and only if 


(13.9) P(A CB) = P(A)P(B). 


If A and B are independent, then P(A | B) = P(A) if P(B) # 0. That is, the conditional 
probability of A, given B, is the same as the “‘absolute’’ probability of A. This relation 
exhibits the significance of independence. The knowledge that B has occurred does not 
influence the probability that A will occur. 
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EXAMPLE |. One card is drawn from a 52-card deck. Each card has the same probability 
of being selected. Show that the two events “drawing an ace” and “drawing a heart’’ are 
independent. 


Solution. We choose a sample space S consisting of 52 elements and assign the point 
probability 5 to each element. The event A, “drawing an ace,” has the probability 
P(A) = & = 73. The event B, “drawing a heart,’”’ has the probability P(B) = $3 = 4. 
The event A © B means “drawing the ace of hearts,’ which has probability 5. Since 


P(A O B) = P(A)P(B), Equation (13.9) is satisfied and events A and B are independent. 


EXAMPLE 2. Three true dice are rolled independently, so that each combination is equally 
probable. Let A be the event that the sum of the digits shown is six and let B be the event 
that all three digits are different. Determine whether or not these two events are independent. 


Solution. Fora sample space S we take the set of all triples (a, 6, c) with a, b, c ranging 
over the values 1, 2, 3, 4, 5, 6. There are 6% elements in S, and since they are equally 
probable we assign the point probability 1/6? to each element. The event A is the set of all 


triples (a, b,c) for which a+ 6-+c=6. Direct enumeration shows that there are 10 
such triples, namely: 


(1, 2, 3), (1, 3, 2), (i, 1, 4), , 4, 2), 
(2, 1, 3), (2, 3, 1), (2, 2, 2), 
(3,1, 2), (3, 2, 1), 


(4, 1,1). 
The event B consists of all triples (a,b,c) for which a #b, bAc, anda#¥c. There 
are 6-5-4 = 120 elements in B. Exactly six of these elements are in set A, so that A 1 B 


has six elements. Therefore 


P(A O B)= 7 : P(A) = a and P(B) = = 


In this case P(A 1 B) ¥ P(A)P(B); therefore events A and B are not independent. 


Independence for more than two events is defined as follows. A finite collection ¥ of 
n events is Said to be independent if the events satisfy the multiplicative property 


(13.10) P(x] = |] P(A,) 
= k=1 
for every finite subcollection {A4,, A.,..., A}, where m may take the values m = 2, 


3,...,, the sets A, being in 7. 
When 7 consists of exactly three events A, B, and C, the condition of independence in 
(13.10) requires that 


(13.11) P(A AB)=P(A)P(B), P(A NC)=P(A)P(C), P(BAC)= P(B)P(C), 
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and 
(13.12) P(A OA BOC) = P(A)P(B)P(C). 


It might be thought that the three equations in (13.11) suffice to imply (13.12) or, in other 
words, that independence of three events is a consequence of independence in pairs. This 
is not true, as one can see from the following example: 

Four tickets labeled a, b, c, and abc, are placed in a box. A ticket is drawn at random, 
and the sample space is denoted by 


S = {a, b,c, abc}. 
Define the events A, B, and C as follows: 
A = {a, abc}, B = {b, abc}, C = {c, abc}. 


In other words, the event X means that the ticket drawn contains the letter x. It is easy 
to verify that each of the three equations in (13.11) is satisfied so that the events A, B, 
and C are independent in pairs. However, (13.12) is not satisfied and hence the three 
events are not independent. The calculations are simple and are left as an exercise for 
the reader. 


13.14 Exercises 
1. Let A and B be two events with P(A) #0, P(B) #0. Show that 


(13.13) P(A © B) = P(B)P(A| B) = P(A)P(B| A). 


Sometimes it is easier to compute the probabilities P(A) and P(B | A) directly by enumeration of 
cases than it is to compute P(A 1 B). When this is the case, Equation (13.13) gives a con- 
venient way to calculate P(A © B). The next exercise is an example. 


2. An urn contains seven white and three black balls. A second urn contains five white and 
five black balls. A ball is selected at random from the first urn and placed in the second. Then 
a ball is selected at random from the second. Let A denote the event “black ball on first draw’”’ 
and B the event “black ball on second draw.”’ 
(a) Compute the probabilities P(A) and P(B| A) directly by enumerating the possibilities. 
Use Equation (13.13) to compute P(A 12 B). 
(b) Compute P(A 1 B) directly by enumerating all possible pairs of drawings. 

3. (a) Let A,, Ay, Ag be three events such that P(A, 1 A,) #0. Show that 


P(A, 0 A, A Ag) = P(Ay)P(Ag| Ay)P(Ag| Ay O Ap). 


(b) Use induction to generalize this result as follows: If A,, 42,..., An, are n events (7 > 2) 
such that P(A, MN Ag A+++ OA An_y) ¥ 0, then 


P(A, \ Ag N-°° 1 An) = P(Ay)P(Ap| Ay)P(As| Ay MO Ag) ++ P(An| Ay O Ao Meee LN An_1)- 


4. A committee of 50 senators is chosen at random. Find the probability that both senators 
from Alaska are included, given that at least one is. 


10. 


13. 


14, 
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. An urn contains five gold and seven blue chips. Two chips are selected at random (without 


replacement). If the first chip is gold, compute the probability that the second is also gold. 


. A deck of cards is dealt into four hands containing 13 cards each. If one hand has exactly 


eight spades, what is the probability that a particular one of the other hands has (a) at least 
one spade? (b) at least two spades? (c) a complete suit? 


. Show that P(A U BIC) = P(A|C) + P(B| C) — P(A NBC). 
. Let Ay, Ag,..., An be n disjoint events whose union Is the entire sample space S. For every 


event E we have the equation 


t=1 t=1 


This equation states that E can occur only in conjunction with some A;. Show that 


(a) P(E) = $ PEN A). 


w=] 


(b) P(E) = > P(E| A)P(AD. 


This formula is useful when the conditional probabilities P(E | A;) are easier to Compute 
directly than P(E). 


. An unbiased coin is tossed repeatedly. It comes up heads on the first six tosses. What is the 


probability that it will come up heads on the seventh toss? 
Given independent events A and B whose probabilities are neither 0 nor 1. Prove that A’ 
and B’ are independent. Is the same true if either of A or B has probability 0 or 1? 


. Given independent events A and B. Prove or disprove in each case that: 


(a) A’ and B are independent. 
(b) A U Band A 2 Bare independent. 
(c) P(A UB) =1 — P(A’)P(B’). 


. If Ay, Ao, ..., An are independent events, prove that 


»(U 4.) + TT Pa) = 1, 
= i=1 


If the three events A, B, and C are independent, prove that A U B and C are independent. 
[Hint: Use the result of Exercise 7 to show that P(A UB | C) = P(A VU B).] 


Let A and B be events, neither of which has probability 0. Prove or disprove the following 
statements: 

(a) If A and B are disjoint, A and B are independent. 

(b) If A and Bare independent, A and B are disjoint. 


. A die is thrown twice, the sample space S consisting of the 36 possible pairs of outcomes 


(a, b) each assigned probability ss. Let A, B, and C denote the following events: 
A = {(a,b)|aisodd}, B = {(a,b)|bis odd}, C = {(a,b)|a + is odd}. 


(a) Compute P(A), P(B), P(C), P(A OB), P(A NC), PIB OC), and P(A N BOC). 
(b) Show that A, B, and C are independent in pairs. 
(c) Show that A, B, and C are not independent. 
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13.15 Compound experiments 


We turn now to the problem of de Méré mentioned in the introduction — whether or not 
it is profitable to bet even money on the occurrence of at least one “double six” in 24 throws 
of a pair of dice. We treat the problem in a more general form: What is the probability of 
throwing a double six at least once in v throws of a pair of dice? Is this probability more 
than one-half or less than one-half when n = 24? 

Consider first the experiment of tossing a pair of fair dice just once. The outcomes of this 
game can be described by ordered pairs (a, 5) in which a and b range over the values 1, 2, 
3,4, 5,6. The sample space S consists of 36 such pairs. Since the dice are fair we assign the 
probability 3’ to each pair in S. 

Now suppose we roll the dice n times. The succession of the ” experiments is one 
compound experiment which we wish to describe mathematically. To do this we need a 
new sample space and a corresponding probability measure. We consider the outcomes of 
the new game as ordered n-tuples (x,;,...,X,), where each component x; is one of the 
outcomes of the original sample space S. In other words, the sample space for the com- 
pound experiment is the n-fold Cartesian product S x --: xX S, which we denote by S”. 
The set S” has 36” elements, and we assign equal probabilities to each element: 


Paes if xeES”. 
36” 


We are interested in the event ‘‘at least one double six in n throws.’’ Denote this event 
by A. In this case it is easier to compute the probability of the complementary event A’, 
which means “‘no double six in n throws.”’ Each element of A’ is an n-tuple whose com- 
ponents can be any element of S except (6, 6). Therefore there are 35 possible values for 
each component and hence (35)” n-tuples altogether in A’. Since each element of A’ has 
probability (s'6)”, the sum of all the point probabilities in A’ is (33)”. This gives us 


P(A) = 1 — P(A’) = 1 — GD". 


To answer de Méré’s question we must decide whether P(A) is more than one-half or less 
than one-half when n = 24. The inequality P(A) > 3 is equivalent to 1 — ($3)" > 3, or 
(33)” <4. Taking logarithms we find 

log 2 


n log 35 — nlog 36 < —log2, or n> ———__ = 24.64 . 
= oe ° log 36 — log 35 


Therefore P(A) < $ when n = 24 and P(A) > 4 whenn > 25. It is not advantageous to 
bet even money on the occurrence of at least one double six in 24 throws. 

The foregoing problem suggests a general procedure for dealing with successive experi- 
ments. If an experiment is repeated two or more times, the result can be considered one 
compound experiment. More generally, a compound experiment may be the result of 
performing two or more distinct experiments successively. The individual experiments may 
be related to each other or they may be stochastically independent, in the sense that the 
probability of the outcome of any one of them is unrelated to the results of the others. 
For the sake of simplicity, we shall discuss how one can combine two independent experi- 
ments into one compound experiment. The generalization to more than two experiments 
will be evident. 
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To associate a bona fide probability space with a compound experiment we must explain 
how to define the new sample space S, the corresponding Boolean algebra -2 of subsets 
of S, and the probability measure P defined on &. As in the above example, we use the 
concept of Cartesian product. 

Suppose we have two probability spaces, say (S,, 4,,P,) and (S,, #, Ps). These 
spaces may be thought of as associated with two experiments £, and £,. By the compound 
experiment E we mean the one for which the sample space S is the Cartesian product 
S, x S,. An outcome of EF is a pair (x, vy) in S, with the first component x an outcome of 
E, and the second component y an outcome of £,. If S, has 1 elements and if S, has m 
elements, then S, x S, has nm elements. 

For the new Boolean algebra & we take the collection of all subsets of S. Next we 
define the probability function P. Since S is finite we can define P(x, ) for each point 
(x, y) in S and then use additivity to define P for subsets of S. The point probabilities 
P(x, y) can be assigned in many ways. However, if the two experiments £, and £, are 
stochastically independent, we define P by the equation 


(13.14) P(x, y) = P,(x)P2(y) for each (x, y)in S. 


Motivation for this definition can be given as follows. Consider two special events A 
and B in the new space S, 


A= {(x1, 1), (x1, Va) ee gy (X15 Vm)} 
and 


B= {(x1, V1), (X25 V1) ee. (X15 Vid} - 


That is, A is the set of all pairs in S, x S, whose first element is x,, and B is the set of all 
pairs whose second element is y,. The intersection of the two sets A and Bis the singleton 
{(x,, ¥,)}. If we feel that the first outcome x, should have no influence on the second out- 
come j, it seems reasonable to require events A and B to be independent. This means we 
would like to define the new probability function P in such a way that we have 


(13.15) P(A 0 B) = P(A)P(B). 


If we decide how to assign the probabilities P(A) and P(B), Equation (13.15) will tell us 
how to assign the probability P(A €M B), that is, the probability P(v,, 1). Event A occurs 
if and only if the outcome of the first experiment is x,. Since P,(x,) is its probability, it 
seems natural to assign the value P,(x,) to P(A) as well. Similarly, we assign the value 
P,(y,) to P(B). Equation (13.15) then gives us 


P(x1,)1) = P,(x,)P2()1)- 


All this, of course, is merely motivation for the assignment of probabilities in (13.14). 
The only way to decide whether or not (13.14) is a permissible assignment of point proba- 
bilities is to check the fundamental properties of probability measures. Each number 
P(x, y) is nonnegative, and the sum of all the point probabilities is equal to 1, since we have 


> P(x,y) = d Pix): D Poy) = 1-1 =1. 


(xz. weS reS} yeSo 
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When we say that a compound experiment E is determined by two stochastically inde- 
pendent experiments £, and E,, we mean that the probability space (S, @, P) is defined in 
the manner just described, “‘independence’’ being reflected in the fact that P(x, v) is the 
product P,(x)P,(y). It can be shown that the assignment of probabilities in (13.14) implies 
the formula 


(13.16) P(U x V) = P,(U)P,(V) 


for every pair of subsets U in &, and V in A. (See Exercise 12 in Section 13.23 for an 
outline of the proof.) We shall deduce some important consequences of this formula. 
Let A be an event (in the compound experiment £) of the form 


A= Cie Ses 


where C, € Z,. Each outcome in A is an ordered pair (x, y) where x is restricted to be 
an outcome of C, (in the first experiment £,) but y can be any outcome of Sy, (in the second 
experiment £,). If we apply (13.16) we find 


P(A) = P(C, X Sy) = P,(C,)P2(S3) = P,(C,), 
since P,(S,) = 1. Thus the definition of P assigns the same probability to A that P, 
assigns to C,. For this reason, such an event A is said to be determined by the first experi- 


ment E,. Similarly, if B is an event of £ of the form 


B= Si x Ca. 
where C, € &,, we have 


P(B) = P(S; X Cy) = P,(S1)Po(C2) = Po( Co) 


and B is said to be determined by the second experiment E,. We shall now show, using 
(13.16), that two such events A and B are independent. That is, we have 


(13.17) P(A (OB) = P(A)P(B). 
First we note that 


AMB= {(x, y)| (x, y)€C, X S, and (x, y) € S; x C3} 
= {(x, y)| x eC, and y € C3} 
= 1C, x C3; 


Hence, by (13.16), we have 
(13.18) P(A CB) = P(C,-X Cy) =P (C)PAG,): 


Since P,(C,) = P(A) and P,(C,) = P(B) we obtain (13.17). Note that Equation (13.18) also 
shows that we can compute the probability P(A  B) as a product of probabilities in the 
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individual sample spaces S, and S,; hence no calculations with probabilities in the 
compound experiment are needed. 

The generalization to compound experiments determined by x experiments £,, F,,..., 
E,, is carried out in the same way. The points in the new sample space are n-tuples 
(X,, X2,...,X,) and the point probabilities are defined as the product of the probabilities 
of the separate outcomes, 


(13.19) P(x), 2 Xn) = P,(%,) P(X) — P,(X,)- 


When this definition of P is used we say that £ is determined by n independent experiments 
E,, E,,..., £,. In the special case in which all the experiments are associated with the 
same probability space, the compound experiment £ is said to be an example of independent 
repeated trials under identical conditions. Such an example is considered in the next 
section. 


13.16 Bernoulli trials 


An important example of a compound experiment was studied extensively by Jakob 
Bernoulli and is now known as a Bernoulli sequence of trials. This is a sequence of repeated 
trials executed under the same conditions, each result being stochastically independent of 
all the others. The experiment being repeated has just two possible outcomes, usually 
called “success” and “failure; the probability of success is denoted by p and that of failure 
by g. Of course, g = 1 —p. The main result associated with Bernoulli sequences is the 
following theorem: 


THEOREM 13.3. BERNOULLI'S FORMULA. The probability of exactly k successes in n 
Bernoulli trials is 


(13.20) (7) par, 


where (") denotes the binomial coefficient, (") = ay eae 
Proof. Denote “‘success” by S and “‘failure’’ by F and consider a particular sequence of 
n results. This may be represented by an n-tuple 


(x1, Xe, eee a 2) 


where each x; is either an S or an F. The event A in which we are interested is the collection 
of all n-tuples that contain exactly k S’sandn — k F’s. Let us compute the point probability 
of a particular n-tuple in A. The probability of each S is p, and that of each Fis q. Hence, 
by (13.19), the probability of each particular n-tuple in A is the product of & factors equal 
to p with n — k factors equal to g. That is, 


Pig kisses) =p a iL “CX Res ote yee Ay 
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Therefore, to compute P(A) we need only count the number of elements in A and multiply 
this number by p*q”"~*. But the number of elements in A is simply the number of ways 
of putting exactly & S’s into the n possible positions of the n-tuple. This is the same as 
the number of subsets of k elements that can be formed from a set consisting of n elements; 
we have already seen that this number is (). Therefore, if we add the point probabilities 
for all points in A we obtain 


P(A) = (;:) p’qr™. 


EXAMPLE |. An unbiased coin is tossed 50 times. Compute the probability of exactly 
25 heads. 


Solution. We interpret this experiment as a sequence of 50 Bernoulli trials, in which 
‘success’’ means “‘heads’”’ and “‘failure’’ means “‘tails.’’ Since the coin is unbiased we assign 
the probabilities p = g = 4, and formula (13.20) gives us (7)(4)°° for the probability of 
exactly k heads in 50 tosses. In particular, when k = 25 we obtain 


5G) “eat 
25)'\9 251251\2) - 


To express this number as a decimal it is best to use logarithms, since tables of logarithms 
of factorials are readily available. If we denote the number in question by P, a table of 
common logarithms (base 10) gives us 


log P = log 50! — 2 log 25! — 50 log 2 
= 64.483 — 50.381 — 15.052 = —0.950 = 0.05 — 1.00 
= log 1.12 — log 10 = log 0.112, 
soP = 0.112. 


EXAMPLE 2. What is the probability of at least r successes in n Bernoulli trials? 


Solution. Let A; denote the event “exactly k successes in n trials.’’ Then the event £ 
in which we are interested is the union 


E=A,UA,,U++*UA,. 


Since the A, are disjoint, we find 


n 


P(E) => ray => (7) pkgr-*. 


k=r 
Since 


n 


2.(;) pq’ *=(p +4)" =1, 


k=0 
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the probability of the complementary event E’ can be computed as follows: 


r—l 
a ee = N\ ke n—k 
P(E’) = 1 — P(E) >i.) 
This last sum gives us the probability of at most r — 1 successes in x trials. 


13.17 The most probable number of successes in n Bernoulli trials 


A pair of fair dice is rolled 28 times. What is the most probable number of sevens? 
To solve this problem we let f(k) denote the probability of exactly & sevens in 28 tosses. 
The probability of tossing a seven is §. Bernoulli’s formula tells us that 


28\. (1 (5\> © 
k) = —) [= 
10 = (i) (6) (5) 
We wish to determine what value (or values) of k in the range k = 0, 1, 2,..., 28 make 


f(k) as large as possible. The next theorem answers this question for any sequence of 
Bernoulli trials. 


THEOREM 13.4. Given an integer n> 1 and a real p,0 <p <_1, consider the set of 
numbers 


f(k) = (;) etc —p)”*, for k=0,1,...,n. 


(a) If (n + 1)p is not an integer, the largest value of f(k) occurs for exactly one k: 
k = [(n + 1)p], the greatest integer < (n + l)p. 
(b) Jf (a + I)p is an integer, the largest value of f(k) occurs for exactly two values of k: 
k =(n+ 1)p and k=(n+1)p-—1. 


Proof. To study the behavior of /(k) we consider the ratio 


MO eet ak 


fork =0,1,...,m—1. The function r(k) is strictly increasing so we have 
0<r(0) <r) <-+'<ra@— 1). 
We consider six cases, illustrated in Figure 13.2. In the first three cases we show that /(k) 


takes its largest value for exactly one k. In the remaining cases f(k) takes its largest value 
for two consecutive values of k. 
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S(k) 


Case 4 Case 5 Case 6 


FiGurE 13.2 Calculation of the most probable number of successes in n Bernoulli 
trials. 


CASE 1. r(Q) > 1. In this case r(k) > 1 for every k so we have 
fO)>f(l) > °° > f(r). 


Therefore the largest value of /(k) occurs only fork = 0. Also, r(0) = (1 — p)/(mp) > 1, 
sol—p>np, (n+ l)p <1, hence [(n + l)p] = 0. 


CASE 2. r(n — 1) <1. In this case r(k) < 1 for every k so f(0) < f(1) < +--+ < f(r) 
and the largest value of f(k) occurs only fork =n. Since r(n — 1) = n(1 — p)/p < 1, we 
have n — np <p, hencen < (n+ I)p << n+ 1,50 [M+ lp] =a. 


CASE 3. r(0) < 1, r(n — 1) > 1, and r(k) ¥ 1 for all k. In this case there is a unique 
integer s,0 <5 <n, such that r(s — 1) < | and r(s) > 1. The function f(x) increases in 
the range 0 < A < s and decreases in the ranges < k <n. Therefore f(k) has a unique 
maximum atk = s. Since r(s — 1) = s(1 — p)/(np — sp + p) < | we haves < (n + l)p. 
The inequality r(s) > 1 shows that (n + ljp <s +1, hence [(n+ l)p] =s. 


Note that in each of the first three cases the maximum value of f(k) occurs when 
k = [(n + l)p]; also (n + 1)p is not an integer in any of these cases. 


CASE 4. r(0) << 1,r(n— 1) > 1, and r(s — 1) = 1 for somes,2<s <n. In this case 
f(k) increases for 0 < k < s — 1 and decreases for s< k <n. The maximum value of 
f(k) occurs twice, when k = s — | and when k = s. The equation r(s — 1) = | implies 
(n+ l)p=s. 


CASE 5. r(n — 1) = 1. In this case r(k) < 1 for k < n — 2, so f(k) increases in the 
range 0 <k C<n—1, and f(n — 1) =f(n). Hence the maximum of /(k) occurs twice, 
when k =n — | and when k =n. The equation r(n — 1) = | implies (n + l)p =n. 
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CASE 6. r(Q) = 1. In this case r(k) > 1 for k > 1, so f(k) decreases in the range 
1<k <n. The maximum /(k) occurs twice, when k = 0 and whenk = 1. The equation 
r(O) = | implies (n + l)p = 


In each of the last three cases the maximum value of /(k) occurs for k = (n + 1)p and 
fork = (n+ l)p — 1. This completes the proof. 


EXAMPLE |. A pair of fair dice is rolled 28 times. What is the most probable number of 
sevens ? 


Solution. We apply Theorem 13.4 with n = 28, p=@, and (n+ Il)p = 22. This is 
not an integer so the largest value of f(k) occurs for k = [22] = 4. 


Note: If the dice are rolled 29 times there are two solututions, k = 4 and k = 5. 


EXAMPLE 2. Find the smallest n such that if a pair of fair dice is thrown n times the 
probability of getting exactly four sevens is at least as large as the probability of getting any 
other number of sevens. 


Solution. We take p = ¢ in Theorem 13.4. We want the largest value of f(k) to occur 
when k = 4. This requires either [((7 + l)p] = 4, (n+ l)p =4, or (n+ Ip—1 =4. 
The smallest n satisfying any of these relations is n = 23. 


13.18 Exercises 


1. A coin is tossed twice, the probability of heads on the first toss being p, and that on the second 
toss pp. Consider this a compound experiment determined by two stochastically independent 
experiments, and let the sample space be 


S = (A, H), (H, T), (T, H), (T, T)}. 


(a) Compute the probability of each element of S. 
(b) Can p, and pz be assigned so that 


P(H, H) = }, P(A, T) = P(T, H) = %, P(T, T) = #4? 
(c) Can p, and p, be assigned so that 
P(H,H)=P(T,T)= 3, P(H,T) =P(T,H) = 68? 
(d) Consider the following four events (subsets of S): 
H,: heads on the first toss, 
H,: heads on the second toss, 
T,: tails on the first toss, 


T, : tails on the second toss. 


Determine which pairs of these four events are independent. 


500 


Set functions and elementary probability 


In each of Exercises 2 through 12 describe your sample space, your assignment of probabilities, 


and 


2. 


the event whose probability you are computing. 


A student takes a true — false examination consisting of 10 questions. He is completely unpre- 
pared so he plans to guess each answer. The guesses are to be made at random. For example, 
he may toss a fair coin and use the outcome to determine his guess. 

(a) Compute the probability that he guesses correctly at least five times. 

(b) Compute the probability that he guesses correctly at least nine times. 

(c) What is the smallest 7 such that the probability of guessing at least n correct answers is 
less than 4? 


. Ten fair dice are tossed together. What is the probability that exactly three sixes occur? 
. A fair coin is tossed five times. What is the probability of getting (a) exactly three heads? 


(b) at least three heads? (c) at most one head? 


. Aman claims to have a divining rod which locates hidden sources of oil. The Caltech Geology 


Department conducts the following experiment to test his claim. He is taken into a room in 
which there are 10 sealed barrels. He is told that five of them contain oil and five contain 
water. His task is to decide which of the five contain oil and which do not. 

(a) What is the probability that he locates the five oil barrels correctly just by chance? 

(b) What is the probability that he locates at least three of the oil barrels correctly by chance? 


. A little old lady from Pasadena claims that by tasting a cup of tea made with milk she can tell 


whether the milk or the tea was added first to the cup. The lady’s claim is tested by requiring 
her to taste and classify 10 pairs of cups of tea, each pair containing one cup of tea made by 
each of the two methods under consideration. Let p denote her “‘true”’ probability of classifying 
a pair of cups correctly. (If she is skillful, p is substantially greater than 3; if not, p <4.) 
Assume the 10 pairs of cups are classified under independent and identical conditions. 

(a) Compute, in terms of p, the probability that she classifies correctly at least eight of the 
10 pairs of cups. 

(b) Evaluate this probability explicitly when p = 3. 


. (Another problem of Chevalier de Méré.) Determine whether or not it is advantageous to 


bet even money on at least one 6 appearing in four throws of an unbiased die. [Hint: Show 
that the probability of at least one 6 in 1 throws is 1 — (3)".] 


. An urn contains w white balls and 6 black balls. Ifk <n, compute the probability of drawing 


k white balls in m drawings, if each ball is replaced before the next one is drawn. 


. Two dice are thrown eight times. Compute the probability that the sum is 11 exactly three 


times. 


. Throw a coin 10 times or 10 coins once and count the number of heads. Find the probability 


of obtaining at least six heads. 


. After a long series of tests on a certain kind of rocket engine it has been determined that in 


approximately 5% of the trials there will be a malfunction that will cause the rocket to misfire. 
Compute the probability that in 10 trials there will be at least one failure. 


. A coin is tossed repeatedly. Compute the probability that the total number of heads will be 


at least 6 before the total number of tails reaches 5. 


. Exercise 12 may be generalized as follows: Show that the probability of at least m successes 


before n failures in a sequence of Bernoullian trials is 


m+n—1 
( + = ) km+n—k—-1 


k=m 


. Determine all 1 with the following property: Ifa pair of fair dice is thrown a times, the prob- 


ability of getting exactly ten sevens is at least as large as the probability of getting any other 
number of sevens. 
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15. A binary slot machine has three identical and independent wheels. When the machine is 
played the possible outcomes are ordered triples (x, y, z), where each of x, y, z can be Oor 1. 
On each wheel the probability of 0 is p and the probability of 1 is 1 —p, whereO <p <1. 
The machine pays $2 if the outcome ts (1, 1, 1) or (0, 0, 0); it pays $1 for the outcome (1, 1, 0); 
otherwise it pays nothing. Let f(p) denote the probability that the machine pays $1 or more 
when it is played once. 
(a) Calculate f(p). 
(b) Define the “payoff” to be the sum Yrs &(x)P(x), where S is the sample space, P(x) is the 
probability of outcome x, and g(x) is the number of dollars paid by outcome x. Calculate the 
value of p for which the payoff is smallest. 


13.19 Countable and uncountable sets 


Up to now we have discussed probability theory only for finite sample spaces. We wish 
now to extend the theory to infinite sample spaces. For this purpose it is necessary to 
distinguish between two types of infinite sets, countable and uncountable. This section 
describes these two concepts. 

To count the members of an n-element set we match the set, element by element, with the 
set of integers {1,2,..., nm}. Comparing the sizes of two sets by matching them element by 
element takes the place of counting when we deal with infinite sets. The process of “‘match- 
ing” can be given a neat mathematical formulation by employing the function concept: 


DEFINITION. Two sets A and B are said to be in one-to-one correspondence if a function f 
exists with the following properties: 
(a) The domain of f is A and the range of f is B. 
(b) If x and y are distinct elements of A, then f(x) and f(y) are distinct elements of B. 
That is, for all x and y in A, 


(13.21) x#y implies f(x) #f(). 


A function satisfying property (13.21) is said to be one-to-one on A. Two sets A and B 
in One-to-one correspondence are also said to be equivalent, and we indicate this by 
writing A ~ B. It is clear that every set A is equivalent to itself, since we may let f(x) = x 
for each x in A. 

A set can be equivalent to a proper subset of itself. For example, the set P = {1, 2, 
3,...}, Consisting of all the positive integers, is equivalent to the proper subset Q = 
{2,4, 6,...} consisting of the even integers. In this case a one-to-one function which 
makes them equivalent is given by f(x) = 2x for x in P. 

If A ~ B we can easily show that B~ A. In fact, if f is one-to-one on A and if the 
range of fis B, then for each b in B there is exactly one ain A such that f(a) = b. Therefore 
we can define an inverse function g on B as follows: If b€ B, g(b) = a, where a is the 
unique element of A such that f(a) = b. This g is one-to-one on B and its range is A; 
hence B~ A. This property of equivalence is known as symmetry: 


(13.22) A~B implies B~A. 
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It is also easy to show that equivalence has the following property, known as transitivity: 
(13.23) A~B and B~C implies A~C. 


A proof of the transitive property is requested in Exercise 2 of Section 13.20. 
A set S is called finite and is said to contain v elements if 


S~ {1,2,...,n}. 


The empty set is also considered to be finite. Sets which are not finite are called infinite sets. 
A set S is said to be countably infinite if it is equivalent to the set of all positive integers, 
that is, if 


(13.24) S~ fl, 2,3,...}. 


In this case there is a function f which establishes a one-to-one correspondence between 
the positive integers and the elements of S; hence the set S can be displayed in roster 
notation as follows: 


S= {f(), f(2),£(3), -- }- 


Often we use subscripts and denote f(k) by a, (or by a similar notation) and we write 
S = {a,, 4, a3,...}. The important thing here is that the correspondence in (13.24) 
enables us to use the positive integers as “labels’’ for the elements of S. 

A set is said to be countable if it is finite or countably infinite. A set which is not countable 
is called uncountable} (Examples will be given presently.) Many set operations when 
performed on countable sets produce countable sets. For example, we have the following 
properties: 

(a) Every subset of a countable set is countable. 

(b) The intersection of any collection of countable sets is countable. 

(c) The union of a countable collection of countable sets is countable. 

(d) The Cartesian product of a finite number of countable sets is countable. 

Since we shall do very little with countably infinite sets in this book, detailed proofs of 
these properties will not be given. Instead, we shall give a number of examples to show 
how these properties may be used to construct new countable sets from given ones. 


EXAMPLE |. The set S of all integers (positive, negative, or zero) is countable. 


Proof. IfneS, let f(n) = 2n if n is positive, and let f(n) = 2 |n| + 1 if mis negative or 
zero. The domain of fis S and its range is the set of positive integers. Since fis one-to-one 
on S, this shows that S is countable. 


EXAMPLE 2. The set R of all rational numbers is countable. 


Proof. For each fixed integer n > 1, let S,, denote the set of rational numbers of the 
form x/n, where x belongs to the set S of Example 1. Each set S,, is equivalent to S [take 


t The words denumerable and nondenumerable are sometimes used as synonyms for countable and un- 
countable, respectively. 
¢ Proofs are outlined in Exercises 3 through 8 of Section 13.20. 
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f(t) = nt if te S,] and hence each S, is countable. Since R is the union of all the S,, 
property (c) implies that R is countable. 


Note. If F = {A,, Ag, As, .. .} is a countable collection of sets, the union of all sets 
in the family ¥ is denoted by the symbols 


U A, or A, VA, UA, U°"*. 
k=1 


EXAMPLE 3. Let A be a countably infinite set, say A = {a,, a2, a3,...}. For each integer 
n> 1, let F, denote the family of n-element subsets of A. That is, let 


F, = {S| S¢ A and Shas n elements}. 
Then each F, is countable. 


Proof. If Sis an n-element subset of A, we may write 


S = {Ay Uys ++ > Me} 
where ky < kp < +++ <k,. Let f(S) = (Q,,a,,...,4,,). That is, f is the function 
which associates with S the ordered n-tuple (a,,, a,,...,4,). The domain of f is 
F, and its range, which we denote by 7,, is a subset of the Cartesian product 
C, = Ax AxX-°:: X A (n factors). Since A is countable, so is C, [by property (d)] and 
hence T,, is also [by property (a)]. But 7,,~ F,, because fis one-to-one. This shows that 
F,, is countable. 


EXAMPLE 4. The collection of all finite subsets of a countable set is countable. 


Proof. The result is obvious if the given set is finite. Assume, then, that the given set 
(call it A) is countably infinite, and let A denote the class of all finite subsets of A: 


F ={S|S¢ Aand S is finite}. 


Then F is the union of all the families 4, of Example 3; hence, by property (c), F is 
countable. 


EXAMPLE 5. The collection of all subsets of a countably infinite set is uncountable. 


Proof. Let A denote the given countable set and let .% denote the family of all subsets 
of A. We shall assume that .e is countable and arrive at a contradiction. If. is countable, 
then < ~ A and hence there exists a one-to-one function f whose domain is A and whose 
range is Z. Thus for each a in A, the function value f(a) is a subset of A. This subset 
may or may not contain the element a. We denote by B the set of elements a such that 
aé f(a). Thus, 

B= {a\|aeA but a¢f(@}. 
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This B, being a subset of A, must belong to the family .7. This means that B = f(b) for 
some b in A. Now there are only two possibilities: (i)b¢€ B, or (ii) b¢ B. If be B, then 
by the definition of B we have b ¢ f(b), which is a contradiction since f(b) = B. There- 
fore (i) is impossible. In case (ii), b € B, which means b ¢ f(b). This contradicts the defini- 
tion of B, so case (ii) is also impossible. Therefore the assumption that . is countable 
leads to a contradiction and we must conclude that © is uncountable. 


We give next an example of an uncountable set that is easier to visualize than that in 
Example 5. 


EXAMPLE 6. The set of real x satisfying 0 < x < 1 is uncountable. 


Proof. Again, we assume the set is countable and arrive at a contradiction. If the set 
is countable we may display its elements as follows: {x,, x2, X3,...}. Now we shall con- 
struct a real number y satisfying 0 < y < 1 which is not in this list. For this purpose we 
write each element x, as a decimal: 


Xn — 0.a,4 ane a,.3 oee 9g 


where each a,, ; is one of the integers in the set {0, 1, 2,...,9}. Let y be the real number 
which has the decimal expansion 
y = 0.) Vo Vg-- +s 
where 
if a,,#l, 


2 if a,, =1. 


Then no element of the set {x,, x2, X3,...} can be equal to y, because y differs from x, 
in the first decimal place, differs from x, in the second decimal place, and in general, y 
differs from x, in the kth decimal place. (A situation like x, = 0.249999--+ and y = 
0.250000 - - - cannot occur here because of the way the y,, are chosen.) Since this y satisfies 
0 < y < 1, we have a contradiction, and hence the set of real numbers in the open interval 
(0, 1) is uncountable. 


13.20 Exercises 


1. Let P = {1,2,3,...} denote the set of positive integers. For each of the following sets, 
exhibit a one-to-one function f whose domain is P and whose range is the set in question: 
(a) A = {2,4,6,...}, the set of even positive integers. 
(b) B = {3, 3?, 3°, ...}, the set of powers of 3. 
(c) C = {2,3,5,7, 11, 13,...}, the set of primes. [Note: Part of the proof consists in 
showing that C is an infinite set. ] 
(d) P x P, the Cartesian product of P with itself. 
(e) The set of integers of the form 23”, where m and 7 are positive integers. 
2. Prove the transitive property of set equivalence: 


If A~B and Bw~cC, then A~C. 


[Hint: Iffmakes A equivalent to Band if g makes B equivalent to C, show that the 
composite function A = g ° f makes A equivalent to C.] 
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Exercises 3 through 8 are devoted to providing proofs of the four properties (a), (b), (c), (d) 
of countable sets listed in Section 13.19. 


3. 


10. 


Prove that every subset of a countable set is countable. [Hint: Suppose S is a countably 
infinite set, say S = {x,, X9,X3,...}, and let A be an infinite subset of S. Let k(1) be the 
smallest positive integer m such that x,,€ A. Assuming k(1), k(2),...,k(m — 1) have been 
defined, let k(n) be the smallest positive integer m > k(n — 1) such that x,,€A. Let f(n) = 
Xiny- Show that fis a one-to-one function whose domain is the set of positive integers and 
whose range is A. This proves the result when S is countably infinite. Construct a separate 
proof for a finite S.] 


. Show that the intersection of any collection of countable sets is countable. [Hint: Use the 


result of Exercise 3.] 


. Let P = {1,2,3,...} denote the set of positive integers. 


(a) Prove that the Cartesian product P x P is countable. [Hint: Let Q denote the set of 
positive integers of the form 23”, where m and nv are positive integers. Then Q © P, so Qis 
countable (by Exercise 3). If (m,n)EP x P, let f(m, n) = 2™3” and use this function to 
show that P x P~@Q.] 

(b) Deduce from part (a) that the Cartesian product of two countable sets is countable. Then 
use induction to extend the result to m countable sets. 


. Let @ = {B,, B,, Bg, ...} be a countable collection of disjoint sets (B; 0 B; = @ wheni # /) 
co 


such that each B, is countable. Show that the union J B, is also countable. [Hint: Let 
fo) k=1 
By, = {1.n> b2.n5b3n>+--$ and S =| B,. If xe S, then x =5b,,, for some unique pair 


k= 
(m, n) and we can define f(x) = (m,n). Use this f to show that S is equivalent to a subset 
of P x P and deduce (by Exercise 5) that S is countable. ] 


. Let SM = {A,, Ag, Ag,...} be a countable collection of sets, and let # = {B,, By, Bz,...} 


be defined as follows: B, = A, and, forn > 1, 


That is, B, consists of those points in A, which are not in any of the earlier sets A,,..., An_y. 
Prove that @ is a collection of disjoint sets (B; © B; = @ when i # /) and that 


This enables us to express the union of any countable collection of sets as the union of a count- 
able collection of disjoint sets. 


. If F¥ isa countable collection of countable sets, prove that the union of all sets in F is count- 


able. [Hint: Use Exercises 6 and 7.] 


. Show that the following sets are countable: 


(a) The set of all intervals on the real axis with rational end points. 

(b) The set of all circles in the plane with rational radii and centers having rational coordinates. 
(c) Any set of disjoint intervals of positive length. 

Show that the following sets are uncountable: 

(a) The set of irrational numbers in the interval (0, 1). 

(b) The set of all intervals of positive length. 

(c) The set of all sequences whose terms are the integers 0 and 1. (Recall that a sequence is 
a function whose domain is the set of positive integers.) 
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13.21 The definition of probability for countably infinite sample spaces 


This section extends the definition of probability to countably infinite sample spaces. 
Let S be a countably infinite set and let Z be a Boolean algebra of subsets of S. We define 
a probability measure P on & as we did for the finite case, except that we require countable 
additivity as well as finite additivity. That is, for every countably infinite collection 
{A,, A,,...} of elements of #, we require that 


(13.25) P(U A,] = > P(A,) if A;NA;= @ whenever i #j. 
k=1 k=1 


Finitely additive set functions which satisfy (13.25) are said to be countably additive (or 
completely additive). Of course, this property requires assuming also that the countable 
union A, U A, U A, U--:: is in & whenever each A, is in &. Not all Boolean algebras 
have this property. Those which do are called Boolean o-algebras. An example is the 
Boolean algebra of all subsets of S. 


DEFINITION OF PROBABILITY FOR COUNTABLY INFINITE SAMPLE SPACES. Let & denote a 
Boolean o-algebra whose elements are subsets of a given countably infinite set S. A Set 
function P is called a probability measure on & if it is nonnegative, countably additive, and 
satisfies P(S) = 1. 


When & is the Boolean algebra of all subsets of S, a probability function is completely 
determined by its values on the singletons (called point probabilities). Every subset A of 
S is either finite or countably infinite, and the probability of A is computed by adding the 
point probabilities for all elements in A, 


P(A) = > P(x). 


The sum on the right is either a finite sum or an absolutely convergent infinite series. 
The following example illustrates an experiment with a countably infinite sample space. 


EXAMPLE. Toss a coin repeatedly until the first outcome occurs a second time; at this 
point the game ends. 

For a sample space we take the collection of all possible games that can be played. This 
set can be expressed as the union of two countably infinite sets A and B, where 


A = {TT, THT, THHT, THHHT,...$ and B= {HH, ATH, ATTH, HTTTH,...}. 
We denote the elements of set A (in the order listed) as ay, a,, a2, ..., and those of set B 


as by, bi, by, .... Wecan assign arbitrary nonnegative point probabilities P(a,) and P(5,,) 
provided that 


¥ Pa,) +  P(b,) = 1, 
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For example, suppose the coin has probability p of coming up heads (//) and proba- 
bility g = 1 — p of coming up tails (7), where 0 < p < 1. Then a natural assignment of 
point probabilities would be 


(13.26) P(a,) = g’p” and P(b,) = pq”. 


This is an acceptable assignment of probabilities because we have 


> Pla) + DP ob = De" + Da 7 en " my ey 


Now suppose we ask for the probability that the game ends after exactly n + 2 tosses. 
This is the event {a,} U {b,}, and its probability is 


P(a,,) + P(b,) = q’p” + p’q”. 


The probability that the game ends in at most n + 2 tosses is 


n n eee po fee. ght} 
> Pay) + > P(by) = q° ——— + p” ee: See ey eee gp"! _ pq”*?, 


13.22 Exercises 
The exercises in this section refer to the example in Section 13.21. 


1, Using the point probabilities assigned in Equation (13.26), let f,(p) denote the probability that 
the game ends after exactly n + 2 tosses. Calculate the absolute maximum and minimum values 
of f,(p) on the interval 0 < p < 1 for each of the values n = 0, 1, 2,3. 

2. Show that each of the following is an acceptable assignment of point probabilities. 


1 
(a) P(a,) = P(b,) ~ pnt2 for n= 0, L254 eee 


1 
(b) P(@n) = Pon) = CG + 3) for 7 eee Os Ge? er ee 
3. Calculate the probability that the game ends before the fifth toss, using: 
(a) the point probabilities in (13.26). 
(b) the point probabilities in Exercise 2(a). 
(c) the point probabilities in Exercise 2(b). 
4. Calculate the probability that an odd number of tosses is required to terminate the game, using: 
(a) the point probabilities in (13.26). 
(b) the point probabilities in Exercise 2(a). 
(c) the point probabilities in Exercise 2(b). 


13.23. Miscellaneous exercises on probability 


1. What is the probability of rolling a ten with two unbiased dice? 

2. Ten men and their wives are seated at random at a banquet. Compute the probability that a 
particular man sits next to his wife if (a) they are seated at a round table; (b) they are seated in 
a row. 
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A box has two drawers. Drawer number | contains four gold coins and two silver coins. 
Drawer number 2 contains three gold coins and three silver coins. A drawer is opened at 
random and a coin selected at random from the open drawer. Compute the probability of each 
of the following events: 

(a) Drawer number 2 was opened and a silver coin was selected. 

(b) A gold coin was selected from the opened drawer. 


. Two cards are picked in succession from a deck of 52 cards, each card having the same prob- 


ability of being drawn. 
(a) What is the probability that at least one is a spade? 

The two cards are placed in a sack unexamined. One card is drawn from the sack and 
examined and found not to be a spade. (Each card has the same probability of being drawn.) 
(b) What is the probability now of having at least one spade? 

The card previously drawn is replaced in the sack and the cards mixed. Again a card is 
drawn and examined. No comparison is made to see if it is the same card previously drawn. 
The card is again replaced in the sack and the cards mixed. This is done a total of three times, 
including that of part (b), and each time the card examined is not a spade. 

(c) What is a sample space and a probability function for this experiment? What is the prob- 
ability that one of the two original cards is a spade? 


. A man has ten pennies, 9 ordinary and | with two heads. He selects a penny at random, 


tosses it six times, and it always comes up heads. Compute the probability that he selected 
the double-headed penny. 


. Prove that it is impossible to load a pair of dice so tnat every outcome from 2 to 12 will have 


the same probability of occurrence. 


. A certain Caltech sophomore has an alarm clock which will ring at the appointed hour with 


probability 0.7. If it rings, it will wake him in time to attend his mathematics class with proba- 
bility 0.8. If it doesn’t ring he will wake in time to attend his class with probability 0.3. 
Compute the probability that he will wake in time to attend his mathematics class. 


. Three horses A, B, and C are ina horse race. The event “‘A beats B’’ will be denoted symboli- 


cally by writing AB. The event “A beats B who beats C”’ will be denoted by ABC, etc. Sup- 
pose it is known that 

P(AB) = 3, P(AC) = 3, P(BC) =3, 
and that 


P(ABC) = P(ACB), P(BAC) = P(BCA), —P(CAB) = P(CBA). 


(a) Compute the probability that A wins. 
(b) Compute the probability that B wins. 
(c) Compute the probability that C wins. 
(d) Are the events AB, AC, and CB independent? 


. The final step in a long computation requires the addition of three integers a,, dg, a3. Assume 


that (a) the computations of a,, a,, and ag are stochastically independent; (b) in the com- 
putation of each a; there is a common probability p that it is correct and that the prob- 
ability of making an error of +1 is equal to the probability of making an error of —1; (Cc) 
no error larger than +1 or less than —1 can occur. Remember the possibility of compensating 
errors, and compute the probability that the sum a, + ag + 4g is Correct. 

The game of “odd man out.” Suppose n persons toss identical coins simultaneously and in- 
dependently, where n > 3. Assume there is a probability p of obtaining heads with each coin. 
Compute the probability that in a given toss there will be an “odd man,” that is, a person whose 
coin does not have the same outcome as that of any other member of the group. 

Suppose x persons play the game ‘“‘odd man out” with fair coins (as described in Exercise 10). 
For a given integer m compute the probability that it will take exactly m plays to conclude the 
game (the mth play is the first time there is an “odd man’). 
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12. Suppose a compound experiment (S, Z, P) is determined by two stochastically independent 
experiments (S,, 4,, P;) and (Sy, B,, Pz), where S = S, x S, and 


P(x, y) = Py(x)P2(y) 
for each (x, y) in S. The purpose of this exercise is to establish the formula 
(13.27) PU x V) = P,(U)P,(V) 


for every pair of subsets U in B, and Vin #,. The sample spaces S, and S, are assumed to 
be finite. 
(a) Verify that Equation (13.27) is true when U and V are singletons, and also when at least 
one of U or V is empty. 

Suppose now that 


U = {u,,ug,...,u,} and abe Pa See Rees eR 


Then U x V consists of the km pairs (u;,v,;). Foreachi = 1,2,...,k let A; denote the set 
of m pairs in U x V whose first component is u;. 

(b) Show that the A; are disjoint sets whose union is U x V. 

(c) Show that 


P(A;) = > PUu;, 0;) = Py(u)P(V). 


j=1 


(d) From (b) and (c) deduce that 


PU xV)= 


k 
P(A;) = P,(U)P(V). 


t=1 
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CALCULUS OF PROBABILITIES 


14.1 The definition of probability for uncountable sample spaces 


A line segment is broken into two pieces, with the point of subdivision chosen at random. 
What is the probability that the two pieces have equal length? What is the probability that 
the longer segment has exactly twice the length of the shorter? What is the probability 
that the longer segment has at least twice the length of the shorter? These are examples of 
probability problems in which the sample space is uncountable since it consists of all 
points on a line segment. This section extends the definition of probability to include 
uncountable sample spaces. 

If we were to use the same procedure as for countable sample spaces we would start with 
an arbitrary uncountable set S and a Boolean o-algebra # of subsets of S and define a 
probability measure to be a completely additive nonnegative set function P defined on # 
with P(S) = 1. As it turns out, this procedure leads to certain technical difficulties that 
do not occur when S is countable. To attempt to describe these difficulties would take us 
too far afield. We shall avoid these difficulties by imposing restrictions at the outset on the 
set S and on the Boolean algebra %. 

First, we restrict S to be a subset of the real line R, or of n-space R”. For the Boolean 
algebra # we use special subsets of S which, in the language of modern integration theory, 
are called measurable subsets of S. We shall not attempt to describe the exact meaning of a 
measurable set; instead, we shall mention some of the properties possessed by the class of 
measurable sets. 

First we consider subsets of R. The measurable subsets have the following properties: 


1. If A is measurable, so is R — A, the complement of A. 

2. If {A,, Ag, Ag,...} is a countable collection of measurable sets, then the union 
A, U A, U Az U>+** is also measurable. 

3. Every interval (open, closed, half-open, finite, or infinite) is measurable. 


Thus, the measurable sets of R form a Boolean o-algebra which contains the intervals. 
A smallest Boolean o-algebra exists which has this property; its members are called Borel 
sets, after the French mathematician Emile Borel (1871-1956). 

Similarly, in 2-space a smallest Boolean o-algebra exists which contains all Cartesian 
products of pairs of intervals; its members are the two-dimensional Borel sets. Borel sets 
in n-space are defined in an analogous fashion. 
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Countability of the set of points with positive probability S11 


Henceforth, whenever we use a set S of real numbers as a sample space, or, more 
generally, whenever we use a set S' in n-space as a sample space, we shall assume that this 
set is a Borel set. The Borel subsets of S themselves form a Boolean o-algebra. These 
subsets are extensive enough to include all the events that occur in the ordinary applications 
of probability theory. 


DEFINITION OF PROBABILITY FOR UNCOUNTABLE SAMPLE SPACES. Let S be a subset of R”, 
and let B& be the Boolean o-algebra of Borel subsets of S.A nonnegative completely additive 
set function P defined on B with P(S) = 1 is called a probability measure. The triple 
(S, &, P) is called a probability space. 


14.2 Countability of the set of points with positive probability 


For countable sample spaces the probability of an event A is often computed by adding 
the point probabilities P(x) for all x in A. This process is not fruitful for uncountable 
sample spaces because, as the next theorem shows, most of the point probabilities are zero. 


THEOREM 14.1. Let (S, @, P) be a probability space and let T denote the set of all x in S 
for which P(x) > 0. Then T is countable. 


Proof. For eachn = 1, 2,3,..., let 7,, denote the following subset of S: 


< P(x) < +h. 
n 


+ 1 


If P(x) > 0 then x € 7, for some n. Conversely, if x € 7, for some n then x € T. Hence 
T=T, UT, U:::. Now 7, contains at most n points, because if there were n + 1 or 
more points in 7, the sum of their point probabilities would exceed 1. Therefore T is 
countable, since it is a countable union of finite sets. 


Theorem 14.1 tells us that positive probabilities can be assigned to at most a countable 
subset of S. The remaining points of S will have probability zero. In particular, if all the 
outcomes of S are equally likely, then every point in S must be assigned probability zero. 


Note: Theorem 14.1 can be given a physical interpretation in terms of mass distri- 
bution which helps to illustrate its meaning. Imagine that we have an amount of mass, with 
the total quantity equal to 1. (This corresponds to P(S) = 1.) Suppose we are able to 
distribute this mass in any way we please along the real line, either by smearing it along 
the line with a uniform or perhaps a varying thickness, or by placing discrete lumps of mass 
at certain points, or both. (We interpret a positive amount of mass as a discrete lump.) 
We can place all the mass at one point. We can divide the mass equally or unequally in 
discrete lumps among two points, among ten points, among a million points, or among 
a countably infinite set of points. For example, we can put $ at 1,4 at2,4at3, and so on, 
with mass (3)” at each integer > 1. Or wecansmear all the mass withoutanyconcentrated 
lumps. Or we can smear part of it and distribute the rest in discrete lumps. Theorem 14.1 
tells us that at most a countable set of points can be assigned discrete lumps of mass. 


Since most (if not all) the point probabilities for an uncountable sample space will be 
zero, a knowledge of the point probabilities alone does not suffice to compute the 
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probabilities of arbitrary events. Further information is required; it is best described in 
terms of two new concepts, random variables and distribution functions, to which we turn 
next. These concepts make possible the use of integral calculus in many problems with 
uncountable sample spaces. Integration takes the place of summation in the computation 
of probabilities. 


14.3 Random variables 


In many experiments we are interested in numbers associated with the outcomes of the 
experiment. For example, n coins are tossed simultaneously and we ask for the number of 
heads. A pair of dice is rolled and we ask for the sum of the points on the upturned faces. 
A dart is thrown at a circular target and we ask for its distance from the center. 
Whenever we associate a real number with each outcome of an experiment we are dealing 
with a function whose domain is the set of possible outcomes and whose range is the set 
of real numbers in question. Such a function is called a random variable. A formal 
definition can be given as follows: 


DEFINITION OF A RANDOM VARIABLE. Let S denote a sample space. A real-valued function 
defined on S is called a one-dimensional random variable. If the function values are ordered 
pairs of real numbers (that is, vectors in 2-space), the function is said to be a two-dimensional 
random variable. More generally, an n-dimensional random variable is simply a function 
whose domain is the given sample space S and whose range is a collection of n-tuples of real 
numbers (vectors in n-space). 


Thus, a random variable is nothing but a vector-valued function defined on a set. The 
term “random’’ is used merely to remind us that the set in question is a sample space.T 

Because of the generality of the above definition, it is possible to have many random 
variables associated with a given experiment. In any particular example the experimenter 
must decide which random variables will be of interest and importance to him. In general, 
we try to work with random variables whose function values reflect, as simply as possible, 
the properties of the outcomes of the experiment which are really essential. 


Notations. Capital letters such as X, Y, Z are ordinarily used to denote one-dimensional 
random variables. A typical outcome of the experiment (that is, a typical element of the 
sample space) is usually denoted by the Greek letter w (omega). Thus, X(w) denotes that 
real number which the random variable X associates with the outcome w. 


The following are some simple examples of random variables. 


EXAMPLE |. An experiment consists of rolling a die and reading the number of points 
on the upturned face. The most “natural’’ random variable X to consider is the one 
stamped on the die by the manufacturer, namely: 


X(w) = @ for w= 1,2, 3,4, 5, 6. 


+ The terms “‘stochastic variable” and “‘chance variable’’ are also used as synonyms for “random variable.”’ 
The word “‘stochastic”’ is derived from a Greek stem meaning “‘chance” and seems to have been invented by 
Jakob Bernoulli. It is commonly used in the literature of probability theory. 
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If we are interested in whether the number of points is even or odd, then we can consider 
instead the random variable Y, which is defined as follows: 


Y(w) = 0 if q@ is even, 


Y(ow) = 1 if qm is odd. 


The values 0 and | are not essential—any two distinct real numbers could be used instead. 
However, 0 and 1 suggest “even’’ and “‘odd,’’ respectively, because they represent the 
remainder obtained when the outcome o 1s divided by 2. 


EXAMPLE 2. A dart is thrown at a circular target. The set of all possible outcomes is 
the set of all points w on the target. If we imagine a coordinate system placed on the 
target with the origin at the center, we can assign various random variables to this experi- 
ment. A natural one is the two-dimensional random variable which assigns to the point w 
its rectangular coordinates (x, y). Another is that which assigns to w its polar coordinates 
(r, 0). Examples of one-dimensional random variables are those which assign to each w 
just one of its coordinates, such as the x-coordinate or the r-coordinate (distance from the 
origin). In an experiment of this type we often wish to know the probability that the dart 
will land in a particular region of the target, for example, the first quadrant. This event can 
be described most simply by the random variable which assigns to each point w its polar 
coordinate angle 0, so that X(w) = 6; the event “‘the dart lands in the first quadrant” is the 
set of w such that 0 < X(mw) < 47. 


Abbreviations. We avoid cumbersome notation by using special abbreviations to describe 
certain types of events and their probabilities. For example, if ¢ is a real number, the set 
of all w in the sample space such that X(w) = ¢ is denoted briefly by writing 


X=t. 


The probability of this event is written P(X = f) instead of the lengthier P({@ | X(w) = ¢}). 
Symbols such as P(X = a or X = b) and P(a < X < b) are defined in a similar fashion. 
Thus, the event ‘“X = a or X = b”’ is the union of the two events “X = a’’ and “X = b”’; 
the symbol P(X =a or X = 5b) denotes the probability of this union. The event 
“a<X <b’ is the set of all points w such that X(w) lies in the half-open interval (a, 5], 
and the symbol P(a < X < 5) denotes the probability of this event. 


14.4 Exercises 


1. Let X be a one-dimensional random variable. 
(a) If a < b, show that the two events a < X < band X < aare disjoint. 
(b) Determine the union of the two events in part (a). 
(c) Show that P(a < X <b) = P(X <b) — P(X <a). 

2. Let (X, Y) denote a two-dimensional random variable defined on a sample space S. This means 
that (X, Y) is a function which assigns to each w in S a pair of real numbers (X(), Y()). 
Of course, each of X and Y is a one-dimensional random variable defined on S. The notation 


X<a,Y<b 
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stands for the set of all elements in S such that X(w) < aand Y(w) < b. 

(a) If a <b and c <d, describe, in terms of elements of S, the meaning of the following 
notation: a<X <b,c< Y<d. 

(b) Show that the twoevents“X <a, Y <c’and"X <a,c < Y < d’”’ are disjoint. Interpret 
these events geometrically. 

(c) Determine the union of the two events in (b). 

(d) Generalize Exercise 1(c) to the two-dimensional case. 

3. Two fair dice are rolled, each outcome being an ordered pair (a, b), where each of a and b is 
an integer from | to 6. Let X be the random variable which assigns the value a + b to the 
outcome (a, b). 

(a) Describe, in roster notation, the events “X =7,’ “X¥ = 11,” “X =7or X = 11.” 
(b) Compute the probabilities of the events in part (a). 

4. Consider an experiment in which four coins are tossed simultaneously (or one coin is tossed 
four times). For each coin define a random variable which assigns the value | to heads and the 
value 0 to tails, and denote these random variables by X,, X,, X3, X,. Assign the probabilities 
P(X; = 1) = P(X; = 0) = 3 for each X;. Consider a new random variable Y which assigns 
to each outcome the total number of heads among the four coins. Express Y in terms of X,, 
X,, X3, X, and compute the probabilities P(Y = 0), P(Y = 1), and P(Y < 1). 

5. Asmall railroad company has facilities for transporting 100 passengers a day between two cities, 
at a fixed cost (to the company) of $7 per passenger. If more than 100 passengers buy tickets 
in any one day the railroad is obligated to provide bus transportation for the excess at a cost of 
$10 per passenger. Let X be the random variable which counts the number of passengers that 
buy tickets ina given day. The possible values of X are the integers 0, 1, 2,3,... up toa certain 
unknown maximum. Let Y denote the random variable which describes the total daily cost 
(in dollars) to the railroad for handling passengers. Express Y in terms of X. 

6. A factory production line consists of two work stations A and B. At station A, X units per 
hour are assembled; they are immediately transported to station B, where they are inspected 
at the rate of Y units per hour, where Y < X. The possible values of X and Y are the integers 
8,9,and 10. Let Z denote the random variable which counts the number of units that come off 
the production line during the first hour of production. 

(a) Express Z in terms of X and Y, assuming each of X and Y is constant during this hour. 
(b) Describe, in a similar way, the random variable U which counts the number of units de- 
livered in the first two consecutive hours of production. Each of X and Y is constant during each 
hour, but the constant values during the second hour need not be the same as those during the 
first. 


14.5 Distribution functions 


We turn now to the problem of computing the probabilities of events associated with 
a given random variable. Let X be a one-dimensional random variable defined on a 
sample space S, where S is a Borel set in n-space for some nm > 1. Let P be a probability 
measure defined on the Borel subsets of S. For each w in S, X(@) is a real number, and 
as w runs through the elements of S the numbers X(w) run through a set of real numbers 
(the range of X). This set may be finite, countably infinite, or uncountable. For each real 
number ¢ we consider the following special subset of S: 


A(t) = {w| X(@) < ¢}. 


If ¢ is less than all the numbers in the range of X, the set A(t) will be empty; otherwise, 
A(t) will be a nonempty subset of S. We assume that for each ¢ the set A(t) is an event, 
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that is, a Borel set. According to the convention discussed at the end of Section 14.3, we 
denote this event by the symbol X < ¢. 

Suppose we know the probability P(X < t) for every real t. We shall find in a moment 
that this knowledge enables us to compute the probabilities of many other events of 
interest. This is done by using the probabilities PLY < t) as a basis for constructing a 
new function F, called the distribution function of X. It is defined as follows: 


DEFINITION OF A DISTRIBUTION FUNCTION. Let X be a one-dimensional random variable. 
The function F defined for all real t by the equation 


F(t) = P(X < t) 
is called the distribution function of the random variable X. 


Note: Sometimes the notation Fx is used to emphasize the fact that the distribution 
function is associated with the particular random variable X. The value of the function 
at ¢ is then denoted by Fx(¢). 


It is important to realize that the distribution function F is defined over the entire real 
axis, even though the range of X may be only a bounded portion of the real axis. In fact, 
if all numbers X (@) lie in some finite interval [a, b], then for t < athe probability P(X < f) 
is zero (since for t < a the set X¥ < ¢ is empty) and for ¢ > 5 the probability PLY < 1) is 1 
(because in this case the set X < ¢ is the entire sample space). This means that for bounded 
random variables X whose range is within an interval [a,b] we have F(t) = 0 for all 
t << aand F(t) = 1 for allt > b. 

We now proceed to derive a number of properties common to all distribution functions. 


THEOREM 14.2. Let F denote a distribution function of a one-dimensional random variable 
X. Then we have: 


(a) O< F(t) <1 for all t. 
(bo) Pa<X<b)=F(b)-F@ ff a<bd. 
(c) F(a) < F(d) if a<b. 


Proof. Part (a) follows at once from the definition of F because probabilities always lie 
between 0 and 1. 


To prove (b) we note that the events “a << X < b” and “X < a” are disjoint. Their 
union is the event “X < 6.” Using additivity we obtain 


Pa<x<bh+P(X<a=P(X < 5d), 
which can also be expressed as 
Pia<X<b)=P(X < b) — P(X < a) = F(b) — Fo. 


Part (c) follows from (b) since Pia < X¥ <b) >0. 


Note: Using the mass analogy, we would say that F(t) represents the total amount of 
mass located between — 00 and ¢ (including the point ¢ itself). The amount of mass located 
in a half-open interval (a, b] is F(b) — F(a). 


516 Calculus of probabilities 


F(t) F(t) 


FiGuRE 14.1 A distribution function Ficure 14.2 A distribution function of an 
of a bounded random variable. unbounded random variable. 


Figure 14.1 shows a distribution function of a bounded random variable X whose 
values X(w) lie in the interval [0,1]. This particular example is known as a uniform 
distribution. Here we have 


F(t)=0 for t <0, F(t)=t for O<t<l, F(t)=1 for t>1. 


Figure 14.2 shows an example of a distribution function corresponding to an unbounded 
random variable. This example is known as a Cauchy distribution and its function values 
are given by the formula 


F(t) = : + a arctan t. 
2 #7 


Experiments that lead to uniform and to Cauchy distributions will be discussed later. 


Note: Using the mass analogy, we would say that in Figure 14.1 no mass has been 
placed to the left of the origin or to the right of the point 1. The entire mass has been 
distributed over the interval [0, 1]. The graph of Fis linear over this interval because the 
mass is smeared with a uniform thickness. In Figure 14.2 the mass has been smeared along 
the entire axis. The graph is nonlinear because the mass has been smeared with a varying 
thickness. 


Theorem 14.2(b) tells us how to compute (in terms of F’) the probability that X lies in a 
half-open interval of the form (a, b]. The next theorem deals with other types of intervals. 


THEOREM 14.3. Let F be a distribution function of a one-dimensional random variable X. 
Then if a <b we have: 

(a) Pia< X < b) = F(b) — F(a) + P(X =). 

(b) Pia < X < b) = F(b) — F(a) — P(X = 5). 

(c) Pa< X <b) = F(b) — F(a) + P(X = a) — P(X = 5B). 


Proof. To prove (a) we note that the events ‘a < X < b” and “X = a” are disjoint and 
their union is “a < X <b.” Using additivity and Theorem 14.2(b) we obtain (a). Parts 
(b) and (c) are similarly proved. 
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Note that all four events 
ax<X<b, a<X<b, ax<X<b, and a<X<b 


have equal probabilities if and only if PLY = a) = 0 and P(X = b) = 0. 


The examples shown in Figures 14.1 and 14.2 illustrate two further properties shared by 
all distribution functions. They are described in the following theorem. 


THEOREM 14.4. Let F be the distribution function of a one-dimensional random variable X. 
Then we have 


(14.1) lim F(t)=0 and lim F(t)=1. 


t+—a@ t+ 00 


Proof. The existence of the two limits in (14.1) and the fact that each of the two limits 
lies between 0 and 1 follow at once, since F is a monotonic function whose values lie between 
0 and 1. 

Let us denote the limits in (14.1) by Z, and L,, respectively. To prove that L, = 0 and 
that L, = | we shall use the countably additive property of probability. For this purpose 
we express the whole space S as a countable union of disjoint events: 


S=U(—-n< X < —n4+1)UU(n< X¥ <n4+1). 


n=1 n=0 


Then, using additivity, we get 


P(S)=SP(—n <X<—nt tt SP(n<X<n4t1) 


= eo n=0 6 
= lim X[F(—n + 1) — F(—n)] + lim 3 [F(n + 1) — Fn). 


The sums on the right will telescope, giving us 
P(S) = lim [F(O) — F(—M)] + lim [F(N + 1) — F(O)] 
M- a N-> 0 


= F(0) — L, + L, — F(0) = L, — Ly. 


Since P(S) = 1, this proves that L, — L, = 1 or L, = 1 + L,. On the other hand, we also 
have L, < 1 and L, > 0. This implies that L, = 0 and L, = 1, as asserted. 


14.6 Discontinuities of distribution functions 


An example of a possible distribution function with discontinuities is shown in Figure 
14.3. Using the mass analogy we would say that F has a jump discontinuity at each point 
which carries a positive amount of mass. As the next theorem shows, the jump is equal to 
the amount of mass concentrated at that particular point. 
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ec REE ieee Jump = P(X = a) oon 


lim F(t) = F(a) 


| {7 a+ 


ae | lim F(t)—p} 


a 


FicureE 14.3 A possible distribution function. FiGureE 14.4 Illustrating a jump dis- 
continuity of a distribution function. 


THEOREM 14.5. Let F be the distribution function of a one-dimensional random variable X. 
Then for each real a we have 


(14.2) lim F(t) = F(a) 
and 
(14.3) lim F(t) = F(a) — P(X = a). 


Note: The limit relation in (14.2) tells us that Fis continuous from the right at each point 
a, because F(t)» F(a) as t > a from the right. On the other hand, Equation (14.3) tells 
us that as f—>a from the left, F(¢) will approach F(a) if and only if the probability 
P(X =a) is zero. When P(X = a) is not zero, the graph of F has a jump discontinuity at 
a of the type shown in Figure 14.4. 


Proof. The existence of the limits follows at once from the monotonicity and bounded- 
ness of F. We prove now that the limits have the values indicated. For this purpose we 
use part (b) of Theorem 14.2. If t > a we write 


(14.4) F(t) = F(a) + Pa<x<t); 
if t < a we write 

(14.5) F(t) = F(a) — P(t< X <a). 
Letting t > a+ in (14.4) we find 


lim F(t) = F(a) +lim P(a < X < 0), 


tat t7at 


whereas if ¢ — a— in (14.5) we obtain 


lim F(t) = F(a) —lim P(t < X <a). 
t t7a— 


Therefore to prove (14.2) and (14.3) we must establish two equations: 


(14.6) limP(a< X<tH=0 


t7a+ 


Discontinuities of distribution functions 519 


and 


(14.7) lim P(t << X < a) = P(X =a). 


t7a— 
These can be justified intuitively as follows: When t—a-+, the half-open interval (a, ¢] 
shrinks to the empty set. That is, the intersection of all half-open intervals (a, t], for 
t >a, is empty. On the other hand, when t + a— the half-open interval (t, a] shrinks 
to the point a. (The intersection of all intervals (t, a] for t < ais the set {a}.) Therefore, 
if probability behaves in a continuous fashion, Equations (14.6) and (14.7) must be valid. 
To convert this argument into a rigorous proof we proceed as follows: 
For each integer n > 1, let 


(14.8) pr=P(a<X<at-), 
n 


To prove (14.6) it suffices to show that p, ~0asn— oo. Let S, denote the event 


1 
n+ 


a+ eS eae ees 
1 n 


The sets S,, are disjoint and their union S, US, US; U:::is theeventa< X <a+tl. 
By countable additivity we have 


8 


(14.9) P(S,) = Pa<X<at+i1)=p,. 
1 


n 


On the other hand, Equation (14.8) implies that 
Pn — Pn4i = P(S,), 


so from (14.9) we obtain the relation 


(14.10) 2 (Pn — Putt) = Pr- 


The convergence of the series is a consequence of (14.9). But the series on the left of 
(14.10) is a telescoping series with sum 


p, — lim p,. 
Therefore (14.10) implies that lim, _.. p, = 0, and this proves (14.6). 
A slight modification of this argument enables us to prove (14.7) as well. Since 


Pt<X<a=P(t<X<at+P(X =a) 


we need only prove that 
limP(t< X <a)=0. 


t~a— 
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For this purpose we introduce the numbers 
1 
an = P(a— 7 <X <a) 
n 


and show that g, —0asn— oo. In this case we consider the events T, given by 


1 
a-—--<X<a-— 
n Ss n+1 


for n= 1, 2, 3, .... These are disjoint and their union is the event a—1< X¥ <a, 
so we have 


> P(T,) = P(a-1<X <a) =q,. 
n=1 
We now note that 9, — ni1 = P(T,,), and we complete the proof as above. 


The most general type of distribution is any real-valued function F that has the following 
properties: 


(a) F is monotonically increasing on the real axis, 
(b) F is continuous from the right at each point, 
(c) lim,,_. F(t) = 0 and lim,_,,. F(t) = 1. 


In fact, it can be shown that for each such function F there is a corresponding set function 
P, defined on the Borel sets of the real line, such that P is a probability measure which 
assigns the probability F(b) — F(a) to each half-open interval (a, b]. For a proof of this 
statement, see H. Cramér, Mathematical Methods of Statistics, Princeton University Press, 
Princeton, N.J., 1946. 

There are two special types of distributions, known as discrete and continuous, that are 
of particular importance in practice. In the discrete case the entire mass is concentrated 
at a finite or countably infinite number of points, whereas in the continuous case the 
mass is smeared, in uniform or varying thickness, along an interval (finite or infinite). 
These two types of distributions will be treated in detail in the next few sections. 


14.7 Discrete distributions. Probability mass functions 


Let X be a one-dimensional random variable and consider a new function p, called the 
probability mass function of X. Its values p(t) are defined for every real number ¢ by the 
equation 

p(t) = P(X =f). 


That is, p(t) is the probability that X takes the value t. When we want to emphasize that 
p is associated with X we write py instead of p and px(t) instead of p(t). 

The set of real numbers ¢ for which p(t) > 0 is either finite or countable. We denote 
this set by T; that is, we let 


T = {t| p(t) > 0}. 


Discrete distributions. Probability mass functions 521 


The random variable X is said to be discrete if 


> p(t) = 1. 


teT 


In other words, X is discrete if a unit probability mass is distributed over the real line by 
concentrating a positive mass p(t) at each point ¢ of some finite or countably infinite set 
T and no mass at the remaining points. The points of T are called the mass points of X. 

For discrete random variables a knowledge of the probability mass function enables 
us to compute probabilities of arbitrary events. In fact, we have the following theorem. 


THEOREM 14.6. Let A be a Borel subset of the real line R, and let P(X € A) denote the 
probability of the set of w such that X(w) € A. Then we have 


(14.11) P(XEA)= D> p(x), 
xeANT 
where T is the set of mass points of X. 
Proof. Since A ATC A and T — AC R — A, we have 


(14.12) 2 (PO <P(XeA) and > p(x) < P(X ¢ A). 


xreT—A 


But A OT and T — A are disjoint sets whose union is 7, so the second inequality in 
(14.12) is equivalent to 


1— > p(x)<1— P(X eA) or > p(x) > P(X eA). 
reA NT rEANT 


Combining this with the first inequality in (14.12) we obtain (14.11). 


Note: Since p(x) = 0 when x € T, the sum on the right of (14.11) can be written as 
yA 4 P(x) without danger of its being misunderstood. 


When A is the interval (— 00, t], the sum in (14.11) gives the value of the distribution 
function F(t). Thus, we have 


F(t) = P(X <1) = ¥ p(x). 


rst 


If a random variable X is discrete, the corresponding distribution function F is also called 
discrete. 
The following examples of discrete distributions occur frequently in practice. 


EXAMPLE 1. Binomial distribution. Let p be a given real number satisfying 0 <p < 1 
and let g = 1 — p. Suppose a random variable X assumes the values 0, 1, 2, ..., n, 
where 7 is a fixed positive integer, and suppose the probability P(XY = k) is given by the 
formula 


PLSD = (;) a" for k=0,1,2,...,n. 
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0.4 


0.2 


0 1 2 3 4 = «5 
(a) The mass function (b) The distribution function 


Figure 14.5 The probability mass function and the distribution function of a 
binomial distribution with parameters n = 5 and p = 4. 


This assignment of probabilities is permissible because the sum of all the point probabilities 


1S 
Spex = k) ->(i) p*q’* =(p+q)"=1. 
k=0 k 


=0 


The corresponding distribution function Fy is said to be a binomial distribution with 
parameters n and p. Its values may be computed by the summation formula 


F x(t) = > (7) p*q”*. 


O<kst 


Binomial distributions arise naturally from a Bernoulli sequence of trials where p is 
the probability of “success”? and g the probability of “failure.”’ In fact, when the random 
variable X counts the number of successes in n trials, P(X = k) is precisely (;)p* q”-* 
because of Bernoulli’s formula. (See Theorem 13.3 in Section 13.16.) Figure 14.5 shows 
the graphs of the probability mass function and the corresponding distribution function 
for a binomial distribution with parameters n = 5 and p = 3. 


EXAMPLE 2. Poisson distribution. Let A be a positive real number and let a random 
variable X assume the values 0, 1, 2, 3,.... If the probability P(Y = k) is given by the 
formula 


—A4k 
Peak 


for k=O 132. ie 


the corresponding distribution function Fy is said to be a Poisson distribution with param- 
eter A. It is so named in honor of the French mathematician S. D. Poisson (1781-1840). 
This assignment of probabilities is permissible because 


[o@) [e6) k 
> P(X =k)= eye =e ‘ei =]. 
ok 


k=0 
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The values of the distribution function are computed from the partial sums 


The Poisson distribution is applicable to many problems involving random events 
occurring in time, such as traffic accidents, connections to wrong numbers in a telephone 
exchange, and chromosome interchanges in cells induced by x-ray radiation. Some specific 
applications are discussed in the books by Feller and Parzen listed at the end of this 
chapter. 


14.8 Exercises 


1. A perfectly balanced die is rolled. For a random variable X we take the function which counts 
the number of points on the upturned face. Draw a graph of the corresponding distribution 
function Fy. 

2. Two dice are rolled. Let X denote the random variable which counts the total number of 
points on the upturned faces. Construct a table giving the nonzero values of the probability 
mass function py and draw a graph of the corresponding distribution function Fy. 

3. The distribution function F of a random variable X is given by the following formulas: 


if t< —2, 
=2:5' <0; 
if O<t <2, 
if ¢>2. 


F(t) = 


a a) 
= 


(a) Sketch the graph of F. 
(b) Describe the probability mass function p and draw its graph. 
(c) Compute the following probabilities: P(X =1), P(X <1), P(X <1), P(X =2), 
P(X <2),P0 <X <2),PO<X <2),P1<X <2). 
4. Consider a random variable X whose possible values are all rational numbers of the form 
n 


n+1 


n+1 
and ——— , where n = 1, 2, 3,... . If 


: on = nti i 
nti.) | oN ntl? 


verify that this assignment of probabilities is permissible and sketch the general shape of the 
graph of the distribution function Fy. 

5. The probability mass function p of a random variable X is zero except at points t = 0, 1, 2. 
At these points it has the values 


p(O) = 3c?, = p(1) = 4c — 10c?, = p(2) = Se — 1, 


forsomec >0Q. 

(a) Determine the value of c. 

(b) Compute the following probabilities: PLY < 1),P(X <2),PQU <X <2),P(0 <X <3). 
(c) Describe the distribution function F and sketch its graph. 

(d) Find the largest ¢ such that F(t) < 4. 

(e) Find the smallest t such that F(t) > 4. 
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6. 


10. 


11. 


12. 
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A random variable X has a binomial distribution with parameters n = 4 and p = }. 
(a) Describe the probability mass function p and sketch its graph. 

(b) Describe the distribution function F and sketch its graph. 

(c) Compute the probabilities P(l < X < 2) and P(l < X <2). 


. Assume that if a thumbtack is tossed on a table, it lands either with point up or in a stable 


position with point resting on the table. Assume there is a positive probability p that it lands 
with point up. 

(a) Suppose two identical tacks are tossed simultaneously. Assuming stochastic independence, 
show that the probability that both land with point up is p?. 

(b) Continuing part (a), let X denote the random variable which counts the number of tacks 
which land with point up (the possible values of X are 0, 1, and 2). Compute the probabilities 
P(X = 0) and P(X = 1). 

(c) Draw the graph of the distribution function Fy when p = 4. 


. Given a random variable X whose possible values are 1,2,...,”. Assume that the prob- 


ability P(X = k)is proportionaltok. Determine the constant of proportionality, the probability 
mass function px, and the distribution function Fy. 


. Given a random variable X whose possible values are 0, 1,2, 3,.... Assume that P(Y = k) 


is proportional to c“/k!, where c is a fixed real number. Determine the constant of proportion- 
ality and the probability mass function p. 

(a) A fair die is rolled. The sample space is S = {1, 2, 3, 4, 5, 6}. If the number of points on 
the upturned face is odd a player receives one dollar; otherwise he must pay one dollar. Let 
X denote the random variable which measures his financial outcome (number of dollars) on 
each play of the game. (The possible values of X¥ are +1 and —1.) Describe the probability 
mass function py and the distribution Fy. Sketch their graphs. 

(b) A fair coin is tossed. The sample space S = {H, T}. If the outcome is heads a player 
receives one dollar; if it is tails he must pay one dollar. Let'Y denote the random variable 
which measures his financial outcome (number of dollars) on each play of the game. Show that 
the mass function py and the distribution F} are identical to those in part (a). This example 
shows that different random variables may have the same probability distribution function. 
Actually, there are infinitely many random variables having a given probability distribution 
F. (Why?) Such random variables are said to be identically distributed. Each theorem 
concerning a particular distribution function is applicable to any of an infinite collection of 
random variables having this distribution. 

The number of minutes that one has to wait for a train at a certain subway station is known 
to be a random variable X with the following probability mass function: 


p(t) =90 unless t = 3k/10 forsome k =0,1,2,...,10. 
P(t) = zs if ¢ = 0, 0.3, 0.6, 0.9, 2.1, 2.4, 2.7, 3.0. 


ptt) =4 if ¢ = 1.2, 1.5, 1.8. 


Sketch the graph of the corresponding distribution function F. Let A be the event that one 
has to wait between 0 and 2 minutes (including 0 and 2), and let B be the event that one has 
to wait between 1 and 3 minutes (including 1 and 3). Compute the following probabilities: 
P(A), P(B), P(A OB), P(B| A), P(A VB). 
(a) If0 <p <1andq =1-—p, show that 


n\ ene — “PY (, _ PY 
(;) o% ar (:-2) Qn, 
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ne 


~ (1 — p)* 


(b) Given 4 > 0, let p = A/nforn > A. Show that Q, —> 1 as n> oo and that 


where 


qk 
(7) p*qr* — x e+ as n>oo. 


This result suggests that for large n and small p, the binomial distribution is approximately 
the same as the Poisson distribution, provided the product np is nearly constant; this constant 
is the parameter 4 of the Poisson distribution. 


14.9 Continuous distributions. Density functions 


Let X be a one-dimensional random variable and let F be its distribution function, so 
that F(t) = P(X < t) for every real ¢. If the probability P(XY = f) is zero for every ¢ then, 
because of Theorem 14.5, F is continuous everywhere on the real axis. In this case F is 
called a continuous distribution and X is called a continuous random variable. If the deriva- 
tive F’ exists and is continuous on an interval [a, t] we can use the second fundamental 
theorem of calculus to write 


(14.13) F(t) — F(a) = | f(u) du, 
where f is the derivative of F. The difference F(t) — F(a) is, of course, the probability 
P(a < X <1), and Equation (14.13) expresses this probability as an integral. 

Sometimes the distribution function F can be expressed as an integral of the form 
(14.13), in which the integrand f is integrable but not necessarily continuous. Whenever 
an equation such as (14.13) holds for all intervals [a, t], the integrand / is called a proba- 


bility density function of the random variable X (or of the distribution F) provided that f is 
nonnegative. In other words, we have the following definition: 


DEFINITION OF A PROBABILITY DENSITY FUNCTION. Let X be a one-dimensional random 
variable with a continuous distribution function F. A nonnegative function f is called a 
probability density of X (or of F) if f is integrable on every interval [a, t] and if 


(14.14) F(t) — F(a) = |" f(u) du. 

If we let a—» —oo in (14.14) then F(a) — 0 and we obtain the important formula 
(14.15) Fa) = P(X <= f' fwdu, 
valid for all real ¢t. If we now let t-» +0 and remember that F(t) — 1 we find that 


(14.16) [> fu) du = 1. 
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For discrete random variables the sum of all the probabilities P(X = t) is equal to 1. 
Formula (14.16) is the continuous analog of this statement There is also a strong analogy 
between formulas (14.11) and (14.15). The density function f plays the same role for con- 
tinuous distributions that the probability mass function p plays for discrete distributions — 
integration takes the place of summation in the computation of probabilities. There is 
one important difference, however. In the discrete case p(t) is the probability that X = ¢, 
but in the continuous case f(t) is not the probability that Y = ¢. In fact, this probability is 
zero because F is continuous for every t. Of course, this also means that for a continuous 
distribution we have 


Pla<X<b)=Paa<X<b=P(a<X¥<b)=P(la<X<b). 


If F has a density f each of these probabilities is equal to the integral [° f(u) du. 


Note: A given distribution can have more than one density since the value of the 
integrand in (14.14) can be changed at a finite number of points without altering the 
integral. But if fis continuous at t then f(t) = F'(t); in this case the value of the density 
function at ¢ is uniquely determined by F. 


Since f is nonnegative, the right-hand member of Equation (14.14) can be interpreted 
geometrically as the area of that portion of the ordinate set of flying to the left of the line 
x =t. The area of the entire ordinate set is equal to 1. The area of the portion of the 
ordinate set above a given interval (whether it is open, closed, or half-open) is the proba- 
bility that the random variable X takes on a value in that interval. Figure 14.6 shows an 
example of a continuous distribution function F and its density function f. The ordinate 
F(t) in Figure 14.6(a) is equal to the area of the shaded region in Figure 14.6(b). 

The next few sections describe some important examples of continuous distributions. 


14.10 Uniform distribution over an interval 


A one-dimensional random variable X is said to have a uniform distribution function F 
over a finite interval [a, b] if Fis given by the following formulas: 


0 if t<a, 
F(t)={(-—" if a<t<b, 

b—a 

1 if t>b. 


f(t) = 1/6 — a) 


FU) =] fw) du 
iy 


a t b a t b 


(a) The distribution function F. (b) The density function fC 


FicureE 14.6 A uniform distribution over an interval [a, b] and the corresponding 
density function. 
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This is a continuous distribution whose graph is shown in Figure 14.6(a). 
The derivative F’(t) exists everywhere except at the points a and b, and we can write 


t 
F(a) =f" flu) du, 
where f is the density function, defined as follows: 


1/(b — if b, 
ro={) a) if a<t< 


0 otherwise. 


The graph of fis shown in Figure 14.6(b). 
The next theorem characterizes uniform distributions in another way. 


THEOREM 14.7. Let X be a one-dimensional random variable with all its values in a finite 
interval [a, b], and let F be the distribution function of X. Then F is uniform over [a, b] if 
and only if 


(14.17) P(X €1) = P(YeJ) 


for every pair of subintervals I and J of [a, b] having the same length, in which case we have 


P(X El= ew : 
b—a 
where h is the length of I. 
Proof. Assume first that XY has a uniform distribution over [a, 5]. If [c,c + A] is any 
subinterval of [a, b] of length h we have 


FO ee a mr Re 3 ao 
b—a b—a b—a 


This shows that P(X €/) = P(X EJ) = h/(b — a) for every pair of subintervals J and J of 
[a, b] of length h. 

To prove the converse, assume that X satisfies (14.17). First we note that F(t) = 0 if 
t<aand F(t)=1 if t > 5, since X has all its values in [a, b]. 

Introduce a new function g defined on the half-open interval (0, b — a] by the equation 


(14.18) g(u) = P(a<X<at+u) if O<u<b—a. 
Using additivity and property (14.17) we find 


gutv)=Pa<c<X¥catutorn) 
=P(ia<Xgqatuyt+Patu<cX¥cat+uty) 
= gu) + Pla< Xf atv) = g(u) + gv), 
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provided thatO <u+v<b-—a. That is, g satisfies the functional equation 
g(u + v) = g(u) + g(v) 
for all u and v such thatu>0,v>0,u+v<b-—a. This is known as Cauchy’s 


functional equation. In a moment we shall prove that every nonnegative solution of Cauchy’s 
functional equation is given by 


g(u) = ——— a(b — a) for O<u<b—a. 
—@ 


Using this in Equation (14.18) we find that for 0 < u < b — awe have 


u 
b—a 


Pa<X<a+u)=——P@<X<b)= 
—a 
since P(a < X < b) =1. In other words, 


F(a + u) — F(a) = ——— if O<u<b—-a. 
—a 


We put ¢ = a + uw and rewrite this as 


t—a 


F(t) — F(a) = —_ 


if a<t<b. 


But F(a) = 0 since F is continuous from the right. Hence 


4 t—a 
F(t) = 
Hee 


if a<t<b, 
which proves that F is uniform on [a, 5]. 


THEOREM 14.8. SOLUTION OF CAUCHY’S FUNCTIONAL EQUATION. Let g be a real-valued 
function defined on a half-open interval (0, c] and satisfying the following two properties: 


(a) g(u + v) = g(u) + g(v) whenever u, v, and u + vare in (0, c], 
and 


(b) g is nonnegative on (0, c]. 


Then g is given by the formula 
2(u) = = g(c) for 0<u<e. 
c 


Proof. By introducing a change of scale we can reduce the proof to the special case in 
which c = 1. In fact, let 


G(x) = g(cx) for O0O<x<l. 
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Then G is nonnegative and satisfies the Cauchy functional equation 
G(x + y) = G(x) + G(y) 
whenever x, y, and x + y are in (0, 1]. If we prove that 
(14.19) G(x) = xG(1) for 0O<x<l 


it follows that g(cx) = xg(c), or that g(u) = (u/c)g(c) forO Cu <e. 
If x is in (0, 1] then x/2 is also in (0, 1] and we have 


G(x) = (3) + (3) =5¢ (5) | 


By induction, for each x in (0, 1] we have 


(14.20) G(x) = na (*) (or ape fe 3 ce, 
n 


Similarly, if y and my are in (0, 1] we have 
G(my) = mG(y) for m=1,2,3,.... 
Taking y = x/n and using (14.20) we obtain 
G (* x] == G(x) 
n n 
if x and mx/n are in (0, 1]. In other words, we have 
(14.21) G(rx) = rG(x) 
for every positive rational number r such that x and rx are in (0, 1]. 
Now take any x in the open interval (0, 1) and let {r,} and {R,} be two sequences of 


rational numbers in (0, 1] such that 


rn<x<R, and such that limr, =limR, =x. 


n> oO n> oO 


Cauchy’s functional equation and the nonnegative property of G show that G(x + y) > 
G(x) so G is monotonic increasing in (0, 1]. Therefore 


G(rn) S G(x) < G(R,). 
Using (14.21) we rewrite this as 
rnG(1) S G(x) < R,G(1). 


Letting n — oo we find xG(1) < G(x) < xG(1), so G(x) = xG(1), which proves (14.19). 
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Note: Uniform distributions are often used in experiments whose outcomes are points 
selected at random from an interval [a, 6], or in experiments involving an interval [a, b] as 
a target, where aiming is impossible. The terms “at random” and “aiming is impossible” 
are usually interpreted to mean that if J is any subinterval of [a, b] then the probability 
P(X € I) depends only on the length of J and not on its location in [a, b]. Theorem 14.7 
shows that uniform distributions are the only distributions with this property. 


We turn now to the probability questions asked at the beginning of this chapter. 


EXAMPLE. A line segment is broken into two pieces, with the point of subdivision chosen 
at random. Let X denote the random variable which measures the ratio of the length of the 
left-hand piece to that of the right-hand piece. Determine the probability distribution 
function Fy. 


Solution. Use the interval [0,1] to represent the line segment and let the point of 
subdivision be described by the random variable Y(w) = w for each w in (0, 1). Since the 
point of subdivision is chosen at random we assume that Y has a uniform distribution 
function Fy over [0, 1]. Hence 


Fy(t)=t for 0<t<1. 


If the segment is broken at w, then w/(1 — «) is the ratio of the length of the left-hand 
piece to that of the right-hand piece. Therefore X(w) = w/(1 — @). 

If t <0 we have F(t) = 0 since the ratio X(w) cannot be negative. If t > 0, the 
inequality X(w) < ¢ is equivalent to w/(l — w) < t, which is equivalent to w < ¢/(1 +f). 
Therefore 


Fx) = P(X <y= P(¥ < 4 — Fy(—5] en ae 


since 0 < ¢/1 +t) <1. 

Now we can calculate various probabilities. For example, the probability that the two 
pieces have equal length is PLY = 1) = 0. In fact, since Fy is a continuous distribution, 
the probability that X takes any particular value is zero. 

The probability that the left-hand segment is at least twice as long as the right-hand 
segment 1s PLY > 2) = 1— P(X < 2)=1— 3% = 3. Similarly, the probability that the 
right-hand segment is at least twice as long as the left-hand segment is PLY < 4) = 4. The 
probability that the longer segment is at least twice as long as the shorter segment is 
P(X > 2) + P(X < 3) = §. 


14.11 Cauchy’s distribution 


A random variable X is said to have a Cauchy distribution F if 


F(t) = : + Z arctan t 


7 
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(a) The distribution function F. (b) The density function f. 


FiGure 14.7 Cauchy’s distribution function and the corresponding density function. 


for all real ¢. This function has a continuous derivative everywhere; a continuous density 
function f is given by the formula 


1 
f(t) “lt 
The graphs of F and f are shown in Figures 14.7(a) and (b), respectively. 

The following experiment leads to a Cauchy distribution. A pointer pivoted at the point 
(—1, 0) on the x-axis is spun and allowed to come to rest. An outcome of the experiment 
is 0, the angle of inclination from the x-axis made by a line drawn through the pointer; 
0 is measured so that —3a < 0 < 37. Let X be the random variable defined by X(@) = 6, 
and let Y be the random variable which measures the y-intercept of the line through the 
pointer. If 0 is the angle described above, then 


Y(0) = tan 0. 
We shall prove that Y has a Cauchy distribution Fy, if X has a uniform distribution over 
[—37, 277 ‘ 


Ifa < tlet « = arctana and let 06 = arctant. Then we have 


= 


Fy(t) — Fy(a) = Pa< ¥<n=Pa<x<H=[ fwau=" 


Since « — —47 as a—» — oo we find 


1 
Pie 7 = “arctan t +. 


T WT 


This shows that Y has a Cauchy distribution, as asserted. 
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14.12 Exercises 


1. A random variable X has a continuous distribution function F, where 


0 if + <0, 
F(t) = lect if Osgt<l, 
1 if ¢>1. 


(a) Determine the constant c and describe the density function ft 
(b) Compute the probabilities PLY = 4), P(X <4), P(\X| <4). 

2. Let f(t) = c |sin ¢| for |t| < 7/2 and let f(t) = 0 otherwise. Determine the value of the constant 
c so that f will be the density of a continuous distribution function F. Also, describe F and 
sketch its graph. 

3. Solve Exercise 2 if f(t) = c(4t — 2t7) for 0 <t <2, and f(t) = 0 otherwise. 

4. The time in minutes that a person has to wait for a bus is known to be a random variable with 
density function f given by the following formulas: 


f(t) =4 for O<rt<l1, /f(t)=4 for 2<t<4, f(t) =0 otherwise. 


Calculate the probability that the time a person has to wait is (a) more than one minute; 
(b) more than two minutes; (c) more than three minutes. 

5. A random variable X has a continuous distribution function F and a probability density f 
The density has the following properties: f(t) = Oift <4,f(4) =1,f()islinearif; <t <3, 
fQ —t) =f (2) for all ¢. 

(a) Make a sketch of the graph of f- 
(b) Give a set of formulas for determining F and sketch its graph. 
(c) Compute the following probabilities: P(X <1), P(X <2), P(X <4), P(X <b, 
P(4 < X <3). 
6. Arandom variable X has a uniform distribution over [—3, 3]. 
(a) Compute P(X = 2), P(X < 2), P(\X| <2), P(X — 2| <2). 
(b) Find a ¢t for which P(X > t) =. 

7. The Lethe Subway Company schedules a northbound train every 30 minutes at a certain 
station. A man enters the station at a random time. Let the random variable X count the 
number of minutes he has to wait for the next train. Assume X has a uniform distribution over 
the interval [0, 30]. (This is how we interpret the statement that he enters the station at 
‘random time.’’) 

(a) For each k = 5, 10, 15, 20, 25, 30, compute the probability that he has to wait at least 
k minutes for the next train. 

(b) A competitor, the Styx Subway Company, is allowed to schedule a northbound train 
every 30 minutes at the same station, but at least 5 minutes must elapse between the arrivals 
of competitive trains. Assume the passengers come into the station at random times and always 
board the first train that arrives. Show that the Styx Company can arrange its schedule so that 
it receives five times as many passengers as its competitor. 

8. Let X be a random variable with a uniform distribution Fy over the interval [0, 1]. Let Y = 
aX +6, where a > 0. Determine the distribution function Fy and sketch its graph. 

9. A roulette wheel carries the integers from 0 to 36, distributed among 37 arcs of equal length. 
The wheel is spun and allowed to come to rest, and the point on the circumference next to a 
fixed pointer is recorded. Consider this point as a random variable X with a uniform distri- 
bution. Calculate the probability that X lies in an arc containing (a) the integer 0; (b) an 
integer n in the interval 11 < m < 20; (c) an odd integer. 
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10. A random variable is said to have a Cauchy distribution with parameters a and b, wherea > 0, 
if its density function is given by 


1 a 


IO Tae E-bF 


Verify that the integral of f from —0o to +00 is 1, and determine the distribution function F. 
11. Let fi(¢) = 1 for 0 <¢ <1, and let f,(¢) = 0 otherwise. Define a sequence of functions { f,} 
by the recursion formula 


fas) = PP. file —t)fp(t)dt. 
(a) Prove that f,4.(x) = J%_, fr(t) dt. 


(b) Make a sketch showing the graphs of f,, f2, and fs. 
12. Refer to Exercise 11. Prove that each function f,, is a probability density. 


14.13 Exponential distributions 


Let A be a positive constant. A one-dimensional random variable X is said to have an 
exponential distribution F with parameter A if 


1 — e4! for t>0, 
F(t) = 
for t <0. 


A corresponding density function fis given by the formulas 


hee for t>0, 


pO ; for t<0O. 


The graphs of F and f are like those shown in Figure 14.8. 


F(t) f(t) 


(a) The distribution function F. (b) The density function f. 
FiGuRE 14.8 An exponential distribution and the corresponding density function. 
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Exponential distributions have a characteristic property which suggests their use in 
certain problems involving radioactive decay, traffic accidents, and failure of electronic 
equipment such as vacuum tubes. This property is analogous to that which characterizes 
uniform distributions and can be described as follows. 

Let X denote the observed waiting time until a piece of equipment fails, and let F be the 
distribution function of X. We assume that F(t) = 0 for t < 0, and for the moment we 
put no further restrictions on F. If t > 0, then X < ¢ is the event “failure occurs in the 
interval [0, ¢].”” Hence X > tis the complementary event, “‘no failure occurs in the interval 
(0, ¢].”’ 

Suppose that no failure occurs in the interval [0, t]. What is the probability of continued 
survival in the interval [t, ¢ + s]? This is a question in conditional probabilities. We wish 
to determine P(X > t +s|X > 1), the conditional probability that there is no failure in 
the interval [0, ¢ + s], given that there is no failure in the interval [0, f]. 

From the definition of conditional probability we have 


(14.22) a a ac ara ee 


Suppose now that F is an exponential distribution with parameter 1 > 0. Then F(t) = 
1—e* for t>0, and P(Y¥>t)=1—P(X <t)=e%'. Hence Equation (14.22) 
becomes 

e Aléts) 


P(X >t+s|X>nH)= =e “= P(X>s). 
e 


—At 


In other words, if the piece of equipment survives in the interval [0, ft], then the probability 
of continued survival in the interval [t, tf + s] is equal to the probability of survival in the 
interval [0, s] having the same length. That is, the probability of survival depends only on 
the length of the time interval and not on the age of the equipment. Expressed in terms of 
the distribution function F, this property states that 


TS ECS). 


(14.23) mare 


1 — F(s) forall t>Oands>0. 


The next theorem shows that exponential distributions are the only probability distribu- 
tions with this property. 


THEOREM 14.9. Let F be a probability distribution function satisfying the functional 
equation (14.23), where F(t) < 1 fort > 0. Then there is a positive constant 4 > 0 such that 


Ft)=1l1—e*  forallt>0. 
Proof. Let g(t) = —log [1 — F(t)] fort > 0. Then | — F(t) = e-®™, so to prove the 


theorem it suffices to prove that g(t) = At for some A > 0. 
Now g is nonnegative and satisfies Cauchy’s functional equation, 


g(t +s) = g(t) + g(s) 
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for allt >0O ands >0. Therefore, applying Theorem 14.8 with c = 1, we deduce that 
g(t) = tg(1) for O << t<l. Let A=g(1). Then A = —log [I — F(1)] > 0, and hence 
g(t) =AtforO0<t<l. 

To prove that g(t) = At for all t > 0, let G(t) = g(t) — At. The function G also satisfies 
Cauchy’s functional equation. Moreover, G is periodic with period 1 because G(t + 1) = 
G(t) + G(1) and G(1) = 0. Since G is identically 0 in (0, 1] the periodicity shows that 
G(t) = Ofor allt > 0. In other words, g(t) = At for all t > 0, which completes the proof. 


EXAMPLE |. Let X be a random variable which measures the lifetime (in hours) of a 
certain type of vacuum tube. Assume X has an exponential distribution with parameter 
A = 0.001. The manufacturer wishes to guarantee these tubes for J hours. Determine T 
so that P(Y > T) = 0.95. 


Solution. The distribution function is given by F(t)=1-—e-*’ for t >0, where 
A=0.001. Since P(X > T) = 1 — F(T) =e", we choose T to make e~*7 = 0.95. 
Hence T = —(log 0.95)/A = —1000 log 0.95 = 51.25+ . 


EXAMPLE 2. Consider the random variable of Example 1, but with an unspecified value 
of A. The following argument suggests a reasonable procedure for determining A. Start 
with an initial number of vacuum tubes at time t = 0, and let g(t) denote the number of 
tubes still functioning ¢ hours later. The ratio [g(0) — g(t)]/g(0) is the fraction of the 
original number that has failed in time ¢. Since the probability that a particular tube fails 
in time ¢ is 1 — e~*’, it seems reasonable to expect that the equation 


(0) — 8) _ gat 
g(0) 


should be a good approximation to reality. If we assume (14.24) we obtain 
g(t) = g(Oje*!. 


In other words, under the hypothesis (14.24), the number g(t) obeys an exponential decay 
law with decay constant 4. The decay constant can be computed in terms of the half-life. 
If t, is the half-life then 4 = g(t,)/g(0) = e-*", so A = (log 2)/t,. For example, if the 
half-life of a large sample of tubes is known to be 693 hours, we obtain A = (log 2)/693 = 
0.001. 


(14.24) 


14.14 Normal distributions 


Let m and o be fixed real numbers, with o > 0. A random variable X is said to have a 
normal distribution with mean m and variance o? if the density function f is given by the 
formula 


f(t) = _ op Ut—m)/a}"/2 
ov 27 


for all real t. The corresponding distribution function F is, of course, the integral 


t 
F(t) = [ eL(u—m)/o}?/2 du. 
— 


1 
oN 2m 


TABLE 14.1 


O(1) = 


1 t 
— | eu"! dy, 
oe —o 


Values of the standard normal distribution function 


0.04 


0.5000 
0.5398 
0.5793 
0.6179 
0.6554 


0.6915 
0.7257 
0.7580 
0.7881 
0.8159 


0.8413 
0.8643 
0.8849 
0.9032 
0.9192 


0.9332 
0.9452 
0.9554 
0.9641 
0.9713 


0.9772 
0.9821 
0.9861 
0.9893 
0.9918 


0.9938 
0.9953 
0.9965 
0.9974 
0.9981 


0.9987 
0.9990 
0.9993 
0.9995 
0.9997 


0.9998 
0.9998 


0.5160 
0.5557 
0.5948 
0.6331 
0.6700 


0.7054 
0.7389 
0.7704 
0.7995 
0.8264 


0.8508 
0.8729 
0.8925 
0.9099 
0.9251 


0.9382 
0.9495 
0.9591 
0.9671 
0.9738 


0.9793 
0.9838 
0.9875 
0.9904 
0.9927 


0.9945 
0.9959 
0.9969 
0.9977 
0.9984 


0.9988 
0.9992 
0.9994 
0.9996 
0.9997 


0.9998 
0.9999 


0.05 


0.5199 
0.5596 
0.5987 
0.6368 
0.6736 


0.7088 
0.7422 
0.7734 
0.8023 
0.8289 


0.8531 
0.8749 
0.8944 
0.9115 
0.9265 


0.9394 
0.9505 
0.9599 
0.9678 
0.9744 


0.9798 
0.9842 
0.9878 
0.9906 
0.9929 


0.9946 
0.9960 
0.9970 
0.9978 
0.9984 


0.9989 
0.9992 
0.9994 
0.9996 
0.9997 


0.9998 
0.9999 


0.06 


0.5239 
0.5636 
0.6026 
0.6406 
0.6772 


0.7123 
0.7454 
0.7764 
0.8051 
0.8315 


0.8554 
0.8770 
0.8962 
0.9131 
0.9279 


0.9406 
0.9515 
0.9608 
0.9686 
0.9750 


0.9803 
0.9846 
0.9881 
0.9909 
0.9931 


0.9948 
0.9961 
0.9971 
0.9979 
0.9985 


0.9989 
0.9992 
0.9994 
0.9996 
0.9997 


0.9998 
0.9999 


0.07 


0.5279 
0.5675 
0.6064 
0.6443 
0.6808 


0.7157 
0.7486 
0.7794 
0.8078 
0.8340 


0.8577 
0.8790 
0.8980 
0.9147 
0.9292 


0.9418 
0.9525 
0.9616 
0.9693 
0.9756 


0.9808 
0.9850 
0.9884 
0.9911 
0.9932 


0.9949 
0.9962 
0.9972 
0.9979 
0.9985 


0.9989 
0.9992 
0.9995 
0.9996 
0.9997 


0.9998 
0.9999 


0.08 


0.5319 
0.5714 
0.6103 
0.6480 
0.6844 


0.7190 
0.7517 
0.7823 
0.8106 
0.8365 


0.8599 
0.8810 
0.8997 
0.9162 
0.9306 


0.9429 
0.9535 
0.9625 
0.9699 
0.9761 


0.9812 
0.9854 
0.9887 
0.9913 
0.9934 


0.9951 
0.9963 
0.9973 
0.9980 
0.9986 


0.9990 
0.9993 
0.9995 
0.9996 
0.9997 


0.9998 
0.9999 


0.09 


0.5359 
0.5753 
0.6141 
0.6517 
0.6879 


0.7224 
0.7549 
0.7852 
0.8133 
0.8389 


0.8621 
0.8830 
0.9015 
0.9177 
0.9319 


0.9441 
0.9545 
0.9633 
0.9706 
0.9767 


0.9817 
0.9857 
0.9890 
0.9916 
0.9936 


0.9952 
0.9964 
0.9974 
0.9981 
0.9986 


0.9990 
0.9993 
0.9995 
0.9997 
0.9998 


0.9998 
0.9999 
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FiGureE 14.9 The standard normal distribu- FIGURE 14.10 The density function of a 
tion function: m =0,o = 1. normal distribution with mean m and 
variance o°. 


It is clear that this function F is monotonic increasing, continuous everywhere, and tends 
to 0 as t—~ —oo. Also, it can be shown that F(t) 1 as t— +0. (See Exercise 7 of 
Section 14.16.) 

The special case m = 0, o = 1 is called the standard normal distribution. In this case 
the function F is usually denoted by the letter ®. Thus, 


t 2 

O(t) = ant el dy. 
a 277 J—0 

The general case can be reduced to the standard case by introducing the change of variable 

vy = (u — m)/o in the integral for F. This leads to the formula 


Fi) = 0(= ™). 


oO 


A four-place table of values of ®(t) for values of ¢ spaced at intervals of length 0.01 is 
given in Table 14.1 for ¢ = 0.00 to ¢ = 3.69. The graph of ® is shown in Figure 14.9. The 
graph of the density fis a famous “‘bell-shaped”’ curve, shown in Figure 14.10. The top of 
the bell is directly above the mean m. For large values of o the curve tends to flatten out; 
for small o it has a sharp peak, as in Figure 14.10. 

Normal distributions are among the most important of all continuous distributions. 
Many random variables that occur in nature behave as though their distribution functions 
are normal or approximately normal. Examples include the measurement of the height 
of people in a large population, certain measurements on large populations of living 
organisms encountered in biology, and the errors of observation encountered when making 
large numbers of measurements. In physics, Maxwell’s law of velocities implies that the 
distribution function of the velocity in any given direction of a molecule of mass M in a 
gas at absolute temperature T is normal with mean 0 and variance M/(kKT), where k is a 
constant (Boltzmann’s constant). 

The normal distribution is also of theoretical importance because it can be used to 
approximate the distributions of many random phenomena. One example is the binomial 
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FicureE 14.11 The density function of a normal distribution considered as an approxi- 
mation to the probability mass function of a binomial distribution. 


distribution with parameters n and p. If X is a random variable having a binomial distri- 
bution with parameters n and p, the probability Pa < X < b) is given by the sum 


b 


» (7) pq", 


k=a 


where g = 1 — p. For a large n, laborious computations are needed to evaluate this sum. 
In practice these computations are avoided by use of the approximate formula 


: = i eee es | 
(14.25) >(:) pig? *§ ~® Pr ) —© ) 
npq npq 


k=a 


where the symbol ~ means that the two sides of (14.25) are asymptotically equal; that is, 
the ratio of the left member to the right member approaches the limit 1 as n> oo. The 
limit relation expressed in (14.25) is a special case of the so-called central limit theorem of 
the calculus of probabilities. This theorem (discussed in more detail in Section 14.30) 
explains the theoretical importance of normal distributions. 

Figure 14.11 illustrates approximate formula (14.25) and shows that it can be accurate 
even for a relatively small value of n. The dotted lines are the ordinates of the probability 
mass function p of a binomial distribution with parameters n = 10 and p= 4%. These 


ordinates were computed from the formula 


MH2PRS i= 8 (5) (5) fom P20, 1, Fah 10: 
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The ordinates for t = 7, 8, 9, and 10 are not shown because their numerical values are 
too near zero. For example, p(10) = (§)!° = 219/10 = 0.0000001024. The smooth 
curve is the graph of the density function fof a normal distribution (with mean m = np = 2 
and variance o® = npg = 1.6). To compute the probability P(a < t < 5) from the mass 
function p we add the function values p(t) at the mass points in the intervala Ct< b. 
Each value p(t) may be interpreted as the area of a rectangle of height p(t) located over 
an interval of unit length centered about the mass point ¢. (An example, centered about 
t = 3, is shown in Figure 14.11.) The approximate formula in (14.25) is the result of 
replacing the areas of these rectangles by the area of the ordinate set of f over the interval 
la—3,b+ 3]. 


14.15 Remarks on more general distributions 


In the foregoing sections we have discussed examples of discrete and continuous distribu- 
tions. The values of a discrete distribution are computed by adding the values of the 
corresponding probability mass function. The values of a continuous distribution with a 
density are computed by integrating the density function. There are, of course, distribu- 
tions that are neither discrete nor continuous. Among these are the so-called “mixed”’ 
types in which the mass distribution is partly discrete and partly continuous. (An example 
is shown in Figure 14.3.) 

A distribution function F is called mixed if it can be expressed as a linear combination 
of the form 


(14.26) F(t) = c,F,(t) + coF,(t), 
where F; is discrete and F, is continuous. The constants c, and c, must satisfy the relations 
0.< ¢< 4, 0<a <1, *+ce¢=1. 


Properties of mixed distributions may be found by studying those that are discrete or 
continuous and then appealing to the linearity expressed in Equation (14.26). 

A general kind of integral, known as the Riemann-Stieltjes integral, makes possible a 
simultaneous treatment of the discrete, continuous, and mixed cases.t Although this 
integral unifies the theoretical discussion of distribution functions, in any specific problem 
the computation of probabilities must be reduced to ordinary summation and integration. 
In this introductory account we shall not attempt to describe the Riemann-Stieltjes integral. 
Consequently, most of the topics we discuss come in pairs, one for the discrete case and one 
for the continuous case. However, we shall only give complete details for one case, leaving 
the untreated case for the reader to work out. 

Even the Riemann-Stieltjes integral is inadequate for treating the most general distribu- 
tion functions. But a more powerful concept, called the Lebesgue-Stieltjes integral, { does 
give a satisfactory treatment of all cases. The advanced theory of probability cannot be 
undertaken without a knowledge of the Lebesgue-Stieltjes integral. 


+ A discussion of the Riemann-Stieltjes integral may be found in Chapter 9 of the author’s Mathematical 
Analysis, Addison-Wesley Publishing Company, Reading, Mass. 1957. 
+ See any book on measure theory. 
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14.16 Exercises 


I. 


10. 


11. 


Let X be a random variable which measures the lifetime (in hours) of a certain type of vacuum 
tube. Assume X has an exponential distribution with parameter 4 = 0.001. Determine T 
so that P(X > T)is (a) 0.90; (b) 0.99. You may use the approximate formula —log (1 — x) = 
x + x?/2 in your calculations. 


. A radioactive material obeys an exponential decay law with half-life 2 years. Consider the 


decay time X (in years) of a single atom and assume that X is a random variable with an 
exponential distribution. Calculate the probability that an atom disintegrates (a) in the interval 
1<X <2; (b) in the interval 2 < X <3; (c) in the interval 2 < X <3, given that it has 
not disintegrated in the interval 0 < X <2; (d) in the interval 2 < X <3, given that it 
has not disintegrated in the interval 1 < X <2. 


. The length of time (in minutes) of long distance telephone calls from Caltech is found to be a 


random phenomenon with probability density function 


ce/3— for ¢t > 0, 


t)= 
ae , for ¢ <0. 


Determine the value of c and calculate the probability that a long distance call will last (a) 
less than 3 minutes; (b) more than 6 minutes; (c) between 3 and 6 minutes; (d) more than 
9 minutes. 


. Given real constants 4 > Oandc. Let 


— if t>c, 


fj)= 
we 0 if t<c. 


Verify that [°,, f(t) dt = 1, and determine a distribution function F having f as its density. 
This is called an exponential distribution with two parameters, a decay parameter 4 and a 
location parameter c. 


. State and prove an extension of Theorem 14.9 for exponential distributions with two parameters 


Aand c. 


. A random variable X has an exponential distribution with two parameters 4 and c. Let Y = 


aX +b,wherea >0. Prove that Y also has an exponential distribution with two parameters 
2’ and c’, and determine these parameters in terms of a, b, c, and A. 


. In Exercise 16 of Section 11.28 it was shown that J e~™* dx = ue n[2. Use this result to prove 


that for o > 0 we have 


ef. ee {-()}a- 


. A random variable X has a standard normal distribution ®. Prove that (a) ®(—x) =1 — 


D(x); (b) P(X] < k) = 20(k) — 13; (©) P(X] > k) = 201 — OK). 


. Arandom variable X has a standard normal distribution ®. Use Table 14.1 to calculate each 


of the following probabilities: (a) P(Y >0); (b) PU <X <2); (c) PUX| <3); @ 
P(|X| > 2). 

A random variable X has a standard norma! distribution ®. Use Table 14.1 to find a number c 
such that (a) P(|X| > c) =4; (b) P(|\X| > c) = 0.98. 

Assume X has a normal distribution function F with mean m and variance o?, and let ® denote 
the standard normal distribution. 
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F(t) = o(—"). 
Oo 


(b) Find a value of c such that P(|X — m| >c) =}. 
(c) Find a value of c such that P(|X — m| > c) = 0.98. 

12. Arandom variable X is normally distributed with mean m = 1 and variance o* = 4. Calculate 
each of the following probabilities: (a) P(—3 < X <3); (b) P(-5 < X < 3). 

13. An architect is designing a doorway for a public building to be used by people whose heights 
are normally distributed, with mean m = 5 ft. 9 in., and variance o? where o = 3 in. How low 
can the doorway be so that no more than 1% of the people bump their heads? 

14. If X has a standard normal distribution, prove that the random variable Y = aX + bis also 
normal if a #0. Determine the mean and variance of Y. 

15. Assume a random variable X has a standard normal distribution, and let Y = X?. 


(a) Prove that 


a ae | 
(a) Show that Fy (t) = sz | e412 dy if ¢>0. 
27 J0 


(b) Determine Fy(t) when t < 0 and describe the density function fy . 


14.17 Distributions of functions of random variables 


If @ is a real-valued function whose domain includes the range of the random variable 
X, we can construct a new random variable Y by the equation 


Y= 9(X), 


which means that Y(w) = p[X(@)] for each w in the sample space. If we know the distribu- 
tion function Fy of X, how do we find the distribution Fy of Y? We begin with an impor- 
tant special case. Suppose that @ is continuous and strictly increasing on the whole real 
axis and takes on every real value. In this case w has a continuous strictly increasing 
inverse wy such that, for all x and y, 


y=(x) ifandonlyif x = p(y). 
By the definition of Fy we have 
Fy(t)=P(Y st) =Pl[p(X) < ¢]. 
Since ¢ is strictly increasing and continuous, the events “g(X) < t” and “X < y(t)” are 


identical. Therefore P[g(X) < t] = P[X < y(t)] = Fx[y(t)]. Hence the distributions Fy 
and Fy are related by the equation 


(14.27) Fy(t) = Fxly(. 


When the distribution Fy and the function py have derivatives we can differentiate both 
sides of (14.27), using the chain rule on the right, to obtain 


Fy() = Fxly()): yO. 
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This gives us the following equation relating the densities: 


fr) =fxlyO)]: vO. 


EXAMPLE 1. Y=aX +b6,a> 0. In this case we have 


2 |= 


p(x) = ax +b, vy) = 2, y'(y) = 


Since is continuous and strictly increasing we may write 


t—b 


Fy() = Fx and fr) = 7 fx(—). 


EXAMPLE 2. Y = X?. In this case w(x) = x* and the foregoing discussion is not directly 
applicable because ¢ is not strictly increasing. However, we can use the same method of 
reasoning to determine Fy and fy. By the definition of Fy we have 


Fy(t) = P(X? <1). 


If t < 0 the event “X? < t” is empty and hence P(X? < t) =0. Therefore F(t) = 0 for 
t<0Q. If t.>0 we have 


P(X? <1) = P(—Vit < X < Vit) = FeV) — Fx(—V 1) + P(X = —V0). 
For a continuous distribution Fy we have P(X¥ = a4) t) = 0 and we obtain the following 
relation between Fy and Fx: 


0 if ¢<0Q, 
Fy(t) = : a 
Few) = FeV). if 10. 


For all t < 0 and for those ¢ > 0 such that Fy 1s differentiable at Vt and at ally we have 
the following equation relating the densities: 


0 if t<0, 
fy) = fxVt) + fx(—V'1t) 
2/1 
Further problems of this type will be discussed in Section 14.23 with the help of two- 
dimensional random variables. 


if t>0. 


14.18 Exercises 


1. Assume X has a uniform distribution on the interval [0, 1]. Determine the distribution function 
F,- and a probability density /;, of the random variable Y if: 
(a) Y=3X +1, (d) Y =log |X|, 
(b) Y= -—3X +1, (e) Y =log xX’, 
(c) Y= X?, (f) Y=ex., 
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2. Let X be a random variable with a continuous distribution function Fy. If y is continuous and 
strictly increasing on the whole real axis and if g(x) > aasx — — and 9(x) > basx > +0, 
determine the distribution function Fy of the random variable Y = y(X). Also, compute a 
density fy , assuming that Fy and ¢ are differentiable. 

3. Assume X has a standard normal distribution. Determine a probability density function of the 
random variable Y when 
(a) Y= X?, (c) Y=e%, 

(b) Y=|X|%, (d) Y = arctan X. 


14.19 Distributions of two-dimensional random variables 


The concept of a distribution may be generalized to n-dimensional random variables 
in a straightforward way. The treatment of the case n = 2 will indicate how the extension 
takes place. 

If X and Y are two one-dimensional random variables defined on a common sample 
space S, (X, Y) will denote the two-dimensional random variable whose value at a typical 
point w of S is given by the pair of real numbers (X(@), Y(w)). The notation 


X<a, Yb 


is an abbreviation for the set of all elements w in S such that X(w) < a and Y(w) <b; 
the probability of this event is denoted by 


P(X <a, Y <b). 


Notations such aa< X<b,c< Y<d, and Pia< X < b,c < Y < d) are similarly 
defined. 

The set of points (x, y) such that x < a and y < b is the Cartesian product A x B of 
the two one-dimensional infinite intervals A = {x|x <a} and B={y|y <b}. The 
set A x B is represented geometrically by the infinite rectangular region shown in Figure 
14.12. The number P(X < a, Y < 5) represents the probability that a point (X(@), Y(@)) 
lies in this region. These probabilities are the two-dimensional analogs of the one-dimen- 
sional probabilities P(X < a), and are used to define two-dimensional probability distribu- 
tions. 


DEFINITION. The distribution function of the two-dimensional random variable (X, Y) 
is the real-valued function F defined for all real a and b by the equation 


F(a, b) = P(X <a, Y <b). 


It is also known as the joint distribution of the two one-dimensional random variables X and Y. 


To compute the probability that (XY, Y) lies in a rectangle we use the following theorem, 
a generalization of Theorem 14.2(b). 
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y 


X= aC Yaad 


X<a,Y¥<c 
Ficure 14.12 An infinite rectangular FiGuRE 14.13 The event “X <b, Y<d” ex- 
region A x B, where A = {x|x <a} pressed as the union of four disjoint events. 


and B={yly <b}. 


THEOREM 14.10. Let F be the distribution function of a two-dimensional random variable 
(XY, Y). Thenifa <bandc < dwe have 


(14.28) Pia<X<b,c< Y<d)= F(b,d) — F(a,d) — F(b,c) + F(a,c). 

Proof. The two events “X¥ <a,c< Y<d” and “X¥ <a, Y<c’” are disjoint, and 
their union is “X < a, Y <d.”’ Adding probabilities we obtain PXY < a,c < Y<d)+ 
P(X <a, Y<c)=P(X <a, Y <d); hence 

P(X a,c < Y<d)= F(a, d) — F(a,c). 
Similarly, we have 
Pia<X<b, Y<c)= F(b,c) — F(a,c). 
Now the four events 
“X <a, VY <c,” “XA < ajc =< Y <a," 


“a<X<b, VY <c,” “a<X<b,c< Y<d” 


are disjoint, and their union is “¥ <b, Y<d.” (See Figure 14.13.) Adding the corre- 
sponding probabilities and using the two foregoing equations we obtain 


F(a, c) + [F(a, d) — F(a, c)] + [F(b,c) — F(a,c] + P(a<X<bc< Y<d) = F(b,d), 


which is equivalent to (14.28). 
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Formula (14.28) gives the probability that the random variable (X, Y) has a value in 
the rectangle (a, b] x (c,d]. There are, of course, corresponding formulas for the rec- 
tangles, [a,b] x [c,d], (a, b) x (c,d), [a, b) x [c, d), and so forth. 

Note: The analogy with mass may be extended to the two-dimensional case. Here the 
total mass 1 is distributed over a plane. The probability Pa < X <b,c < Y <d) 
represents the total amount of mass located in the rectangle (a, b] x (c, d]. The number 
F(a, b) represents the amount in the infinite rectangular region X <a, Y <5. As in 
the one-dimensional case, the two most important types of distributions are those known as 
discrete and continuous. In the discrete case the entire mass is located in lumps concen- 
trated at a finite or countably infinite number of points. In the continuous case the mass 
is smeared all over the plane with a uniform or varying thickness. 


14.20 Two-dimensional discrete distributions 


If a random variable (X, Y) is given we define a new function p, called the probability 
mass function of (XY, Y), such that 


P(x, y) = P(X =x, Y=y) 


for every pair of real numbers (x, y). Let T denote the set of (x, y) for which p(x, y) > 0. 
It can be shown that 7 is either finite or countably infinite. If the sum of the p(x, y) for 
all (x, y) in T is equal to 1, that is, if 


(14.29) YP y=, 


(x,y)eE 


the random variable (X, Y) is said to be discrete (or jointly discrete). The points (x, y) 
in T are called the mass points of (X, Y). 


Suppose that x,, X2, X3,... and y,, Ye, yg, -.- are among the possible values of XY and 
Y, respectively, and let 


Pixs = P(X = x;, Y = y;). 


If each p,,; is positive and if the sum of all the p,; is 1, then the probability of an event 
“(X, Y)e£” is the sum of all the p,; taken over all x; and y; for which (x,;, y;)¢ £. We 
indicate this by writing 
P[(X, Y)€ E] = pap Pis- 
te yer 
In particular, since P(X < x, Y < y) = F(x, y), the joint distribution F (which is also 
called discrete) is given by the double sum 


(14.30) F(x,y)= > Dd Dis. 


2;<2 Vis 


The numbers p;; can also be used to reconstruct the probability mass functions px 
and py of the one-dimensional random variables X and Y. In fact, if E;; denotes the 
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event “XY = x,;, Y=y;,” the events E;,, E;2, E;3, ... are disjoint and their union is the 
event “X = x,.”’ Hence, by countable additivity, we obtain 


(14.31) P(X = x;) = 2 PEs) — 2 Pij- 
j= j= 

Similarly, we have 

(14.32) P(Y = yj) = 2 P(Eis) = 2 Pi;- 


Therefore, the corresponding one-dimensional distributions Fy and Fy can be computed 
from the formulas 


Fx(t) = > P(X = x,;) = > > Pi 
azSt 2;<t j=1 


and 


Fy) = 2 PY =y) => Yu. 


yj;<ti=1 


For finite sample spaces, of course, the infinite series are actually finite sums. 


14.21 Two-dimensional continuous distributions. Density functions 


As might be expected, continuous distributions are those that are continuous over the 
whole plane. For the majority of continuous distributions F that occur in practice there 
exists a nonnegative function f (called the probability density of F) such that the proba- 
bilities of most events of interest can be computed by double integration of the density. 
That is, the probability of an event “(X, Y) € Q”’ is given by the integral formula 


(14.33) PUX, ye Ql= |] f 
Q 


When such an f exists it is also called a probability density of the random variable (X, Y), 
or a joint density of X and Y. We shall not attempt to describe the class of regions Q for 
which (14.33) is to hold, except to mention that this class should be extensive enough to 
include all regions that arise in the ordinary applications of probability. For example, if a 
joint density exists we always have 


(14.34) Pa<X<bc<¥<ad=|| f(x,y) dxdy, 
R 


where R = [a, b] x [c,d]. The integrand / is usually sufficiently well behaved for the 
double integral to be evaluated by iterated one-dimensional integration, in which case 
(14.34) becomes 
d b b d 
Pa<X<be<y¥<a=I" [lp ax] ay=f? [Fe yay] ax. 


In all the examples we shall consider, this formula is also valid in the limiting cases in 
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which a and c are replaced by —oo and in which b and d are replaced by +0. Thus 
we have 


(14.35) Fb a= im | [. f(x, y)dx| dy = ie [ i f(x, y) dy| dx 
for all 6 and d, and 


(14.36) [- | im f(x, y) dx| dy = [- Lf” poe, y) dy] dx = 1. 


— 0 


Equations (14.35) and (14.36) are the continuous analogs of (14.30) and (14.29), 
respectively. 

If a density exists it is not unique, since the integrand in (14.33) can be changed at a 
finite number of points without affecting the value of the integral. However, there is at 


most one continuous density function. In fact, at points of continuity of f we have the 
formulas 


I(x, y) = Dy 2F(x, y) = DoiF(x, y), 


obtained by differentiation of the integrals in (14.35). 
As in the discrete case, the joint density f can be used to recover the one-dimensional 
densities fx and fy. The formulas analogous to (14.31) and (14.32) are 


+00 +00 
f= faydy and fpo=], fe ya. 


The corresponding distributions Fy(t) and Fy(t) are obtained, of course, by integrating the 
respective densities fy and fy from — © tot. 

The random variables X and Y are called independent if the joint distribution F(x, y) can 
be factored as follows, 


F(x, y) = Fx(x)Fy(y) 
for all (x, y). Some consequences of independence are discussed in the next set of exercises. 


EXAMPLE. Consider the function f that has the constant value | over the square R = 
(0, 1] x [0, 1], and the value 0 at all other points of the plane. A random variable (XY, Y) 
having this density function is said to be uniformly distributed over R. The corresponding 
distributions function F is given by the following formulas: 

xy if (x,y)ER, 

x if 0<x<l and y>l, 
F(x,y)=(y if O<y<l and x>l, 

l if x>1 and Diels 

0 otherwise. 


The graph of F over R is part of the saddle-shaped surface z = xy. At all points (x, y) not 
on the boundary of R the mixed partial derivatives D, F(x, y) and D, ,F(x, y) exist and 
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are equal to f(x, y). This distribution is the product of two one-dimensional uniform 
distributions Fy and Fy. Hence X and Y are independent. 


14.22 Exercises 


1. Let X and Y be one-dimensional random variables with distribution functions Fy and Fy, 
and let F be the joint distribution of X and Y. 
(a) Prove that X and Y are independent if and only if we have 


Pia<X <bc< Y<d)=P(a<X <b)P(c < Y <d) 


for alla, b,c, d, witha < bande <d. 

(b) Consider the discrete case. Assume x,,%2,... and y,, yo,... are the mass points of X 
and Y, respectively. Let a; = P(X = x;) and b; = P(Y =y,). If pj; = P(X =x;, Y =y,), 
show that X and Y are independent if p;; = a;b; for all i and /. 

(c) Let X and Y have continuous distributions with corresponding densities fy and fy and 
let f denote the density of the joint distribution. Assume the continuity of all three densities. 
Show that the condition of independence is equivalent to the statement f(x, y) =fx(x) fy (y) 
for all (x, y). [Hint: Express fas a derivative of the joint distribution F.] 

2. Refer to Exercise 1. Suppose that P(X = x,, Y = y,) = P(X = x2, Y = y.) = p/2 and that 
P(X = x1, Y = yo) = P(X = x2, Y = yy) ='g/2, where p and g are nonnegative with sum 1. 
(a) Determine the one-dimensional probabilities P(X = x,) and P(Y = y,) for i = 1, 2 and 
j =1,2. 

(b) For what value (or values) of p will X and Y be independent? 

3. Ifa < bande < d, define fas follows: 


1 


(ENAo=od=o. 8 CORON 
0 


otherwise. 


(a) Verify that this is the density of a continuous distribution F and determine F. 
(b) Determine the one-dimensional distributions Fy and Fy. 
(c) Determine whether or not X and Y are independent. 
4. If P(Y < b) #0, the conditional probability that X < a, given that Y <b, is denoted by 
P(X <a|Y <5), and is defined by the equation 


> P P(X <a, Y <b) 

(X <al| Y<b)= PCY <b) 

If P(Y < b) =0, we define P(X < a| Y <b) =P(X <a). Similarly, if P(X < a) #0, 
we define P(Y <b|X <a) = P(X <a, Y < b)/P(X <a). If P(X <a) =0, we define 
P(Y <b|X <a) =P(Y <b). 

(a) Refer to Exercise 1 and describe the independence of X and Y in terms of conditional 
probabilities. 

(b) Consider the discrete case. Assume x,, %2,... and y,, yo,... are the mass points of X 
and Y, respectively. Show that 


P(X =x,;) = > P(Y = y) P(X =x; | Y = y,) 
ad 
and 


P(Y =y,) = ¥ P(X = x)P(Y = y,| X = X;). 
i=1 


10. 
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. A gambling house offers its clients the following game: A coin is tossed. If the result of the 


first throw is tails, the player loses and the game is over. If the first throw is heads, a second 
throw is allowed. If heads occur the second time the player wins $2, but if tails comes up the 
player wins $1. Let X be the random variable which is equal to 1 or 0, according to whether 
heads or tails occurs on the first throw. Let Y be the random variable which counts the number 
of dollars won by the player. Use Exercise 4 (or some other method) to compute P(Y = 0), 
P(Y = 1), and P(Y = 2). 


. Refer to Exercise 4. Derive the so-called Bayes’ formulas: 


_ _ PX = x,)P(Y = yj | X = xx) 
P(X = x, | ¥ = yj) = Dita POX = x)P(Y = yj |X = x)’ 
P(Y = y,)P(X = x;| Y = yz) 


POSSESS Sa ay yr a Y=) 


. Given two urns A and B. Urn A contains one $5 bill and two $10 bills. Urn B contains three 


$5 bills and one $10 bill. Draw a bill from urn A and put it in urn B. Let Y be the random 
variable which counts the dollar value of the bill transferred. Now draw a bill from urn B 
and use the random variable X to count its dollar value. Compute the conditional probabilities 


P(Y =5|X = 10) and P(Y = 10| X = 10). 


[Hint: Use Bayes’ formulas of Exercise 6.] 


. Given three identical boxes, each containing two drawers. Box number | has one gold piece 


in one drawer and one silver piece in the other. Box 2 has one gold piece in each drawer and 
Box 3 has one silver piece in each drawer. One drawer is opened at random and a gold piece 
is found. Compute the probability that the other drawer in the same box contains a silver piece. 
[Hint: Use Bayes’ formulas of Exercise 6.] 


. Let Q be a plane region with positive area a(Q). A continuous two-dimensional random 


variable (X, Y) is said to have a uniform distribution over Q if its density function fis given by 
the following formulas: 


_{l/a(Q) if &, ed, 
fon ={, if (x,y) ¢Q. 


(a) If Eis a subregion of Q with area a(E), show that a(E)/a(Q) is the probability of the event 
(X, YEE. 

(b) Raindrops fall at random on the square Q with vertices (1,0), (0, 1), (—1, 0), (0, —1). 
An outcome is the point (x, y) in Q struck by a particular raindrop. Let X(x, y) = x and 
Y(x, y) = y and assume (X, Y) has a uniform distribution over Q. Determine the joint 
density function f and the one-dimensional densities fy and fy. Are the random variables X 

and Y independent? 


A two-dimensional random variable (X, Y) has the joint distribution function F. Let U = 
X —a, V = Y —b, where a and b are constants. If G denotes the joint distribution of 
(U, V) show that 

G(u,v) = Fu +a,v + bd). 


Derive a similar relation connecting the density function f of (X, Y) and g of (U, V) when fis 
continuous. 
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14.23 Distributions of functions of two random variables 


We turn now to the following problem: If X and Y are one-dimensional random 
variables with known distributions, how do we find the distribution, of new random variables 
such as X + Y, XY, or X2 + Y?? This section describes a method that helps to answer 
questions like this. Two new random variables U and V are defined by equations of the 


form 
U= M(X, Y), V = NX, Y), 


where M(X, Y) or N(X, Y) is the particular combination in which we are interested. From 
a knowledge of the joint distribution f of the two-dimensional random variable (X, Y) 
we calculate the joint distribution g of (U, V). Once g is known, the individual distribu- 
tions of U and V are easily found. 

To describe the method in detail, we consider a one-to-one mapping of the xy-plane 
onto the uv-plane defined by the pair of equations 


u= M(x, y), v = N(x, y). 


Let the inverse mapping be given by 
=Q(u,v), y= Rly, v), 


and assume that Q and R have continuous partial derivatives. If T denotes a region in the 
xy-plane, let 7’ denote its image in the uv-plane, as suggested by Figure 14.14. Let X and 
Y be two one-dimensional continuous random variables having a continuous joint distribu- 
tion and assume (X, Y) has a probability density function f. Define new random variables 
U and V by writing U = M(X, Y), V = N(X, Y). To determine a probability density g 
of the random variable (U, V) we proceed as follows: 

The random variables X and Y are associated with a sample space S. For each w in S$ 
we have U(w) = M[X(m), Y(w)] and V(w) = N[X(@), Y(w)]. Since the mapping is 
one-to-one, the two sets 


{o|(U@), Vo))eT} and — {| (X(), ¥()) € T} 
are equal. Therefore we have 
(14.37) P((U, V)€T’] = P{(X, Y)eET]. 


Since f is the density function of (Y, Y) we can write 
(14.38) PUX, YET] = || f(x, y) dx dy. 
T 


Using (14.37) and the formula for transforming a double integral we rewrite (14.38) as 
follows: 


du dv. 


P[(U, Y)€T’] = l f[O(u, v), R(u, v)] aaa . 


bd 
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FiGureE 14.14 A one-to-one mapping of a region T in the xy-plane onto a region T” in 
the uv-plane. 


Since this is valid for every region JT’ in the wv-plane a density g of (U, V) is given by the 
integrand on the right; that is, we have 


o(Q, R) 


O(u, v) 


(14.39) g(u, v) = f[Q(u, v), R(u, v)] 


The densities fy and f, can now be obtained by the integration formulas 


fu = [. g(u,v)dv, f= [. g(u, v) du. 


EXAMPLE |. The sum and difference of two random variables. Given two one-dimensional 
random variables X and Y with joint density f, determine density functions for the random 
variable U= ¥ + YandV=X— Y. 


Solution. We use the mapping given byu = x+y,v=x—y. This is a nonsingular 
linear transformation whose inverse is given by 


ca "F amy, yatS Pawo, 
The Jacobian determinant is 
og 02) |1 1 
a(Q, R) _ Ou dv 2 2} 1 
Que) }aR OR) |1 _1f 2 
Ou Ov 2 2 
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Applying Equation (14.39) we see that a joint density g of (U, V) is given by the formula 
_1l,futov uv 

=>! ( oD ) 


To obtain a density fy = /x,y we integrate with respect to v and find 


fav = 5) (2) dv. 


a(Q, R) 
O(u, v) 


g(u, 0) =s(* 3 Fa 


The change of variable x = 4(u + v), dx = $ dv, transforms this to 


fxiy(u) = [° fe u — x) dx. 


Similarly, we find 


fx-r(v) = i sr eS) au = [Eteax = oar. 


An important special case occurs when X and Y are independent. In this case the joint 
probability density factors into a product, 


F(x, y) =fxQOfrQ), 


and the integrals for fy,» and fxy_y become 


fev) =] fxr — dx, fev) =|" feofyle — 0) dx. 


EXAMPLE 2. The sum of two exponential distributions. Suppose now that each of X and Y 
has an exponential distribution, say fy(t) = fy (t) = 0 for t < 0, and 
fx(t)=de“*, fy(t)=ue" for t>0. 
Determine the density of X¥ + Y when X and Y are independent. 


Solution. If u <0 the integral for fy,y(u) is 0 since the factor fy(x) = 0 for x < 0, 
and the factor fy, (u — x) =Oforx >0. Ifu > 0 the integral for fy, »(u) becomes 


faery (u) = {° he ye Be? dx = Awe*™ ik elt Ale dx. 


To evaluate the last integral we consider two cases, w= Aand uw # A. 
If ~ = A the integral has the value u and we obtain 


fxsy(u) = Pue™ for u>O0O. 
If u # A we obtain 


—AU _ pg -hu 
a ——— for u>O0. 


ae elt-Alu ae | 
Sx+y(u) = Ape ———— 
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EXAMPLE 3. The maximum and minimum of two independent random variables. Let X and 
Y be two independent one-dimensional random variables with densities fy and fy and 
corresponding distribution functions Fy and Fy. Let U and V be the random variables 


U = max {X, Y}, V = min {X, Y}. 


That is, for each w in the sample space, U(w) is the maximum and V(w) is the minimum of 
the two numbers X(w), Y(w). The mapping uv = max {x, y}, v = min {x, y} is not one- 
to-one, so the procedure used to deduce Equation (14.39) is not applicable. However, in 
this case we can obtain the distribution functions of U and V directly from first principles. 

First we note that U <1¢ if, and only if, X << t and Y<t. Therefore P(U < t) = 
P(X <t, Y<t). By independence this is equal to P(X < t)P(Y < t) = Fx(t)F y(t). 
Thus, we have 

Fy(t) = Fx(t)Fy(t). 


At each point of continuity of fy and fy we can differentiate this relation to obtain 


fu(t) = fx()Fr(t) + Fx(Ofy(). 
Similarly, we have V > tif and only if ¥ >tand Y>t. Therefore 


FAth=PV<th=1—-PV>tH)=1-P(X>t,Y>H)=1—-P(X>D)P(Y >?) 
=1—(1 — Fx(t)) — Fy(t)) = Fx(t) + Fy (t) — Fe(OF yp (0). 


At points of continuity of fy and fy we differentiate this relation to obtain 
Sv O =fx) + fe (0 — fxOFr() — FxeOfr (). 


14.24 Exercises 


1. Let X and Y be two independent one-dimensional random variables, each with a uniform 
distribution over the interval [0,1]. Let U = X + YandletV =X — Y. 
(a) Prove that U has a continuous density fy given by 


u if O<u<l, 
fut) =(2—-u if l<u<2, 


0 otherwise. 


(b) Describe, in a similar way, a continuous density fy for V. 
(c) Determine whether or not U and V are independent. 
2. Let X and Y be as in Exercise 1, and let U = max {X, Y}, V = min {X, Y}. 
(a) Prove that U has a density function such that fy(¢) = 2 forO ¢¢ <1, and fy(t) =0 
otherwise. 
(b) Describe a density function fp for V. 
(c) Determine whether or not U and V are independent. 
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3. Let Xand Y be two independent one-dimensional random variables, each having an exponential 


6. 


distribution with parameter A = 1, andlet f(x, y) = fx(x) fy (), the product of the densities of 
X and Y. 

(a) Let A denote the set of points in the xy-plane at which f(x, y) > 0. Make a sketch of 
A and of its image A’ under the mapping defined by u = x + y,v =x/(x + y). 

(b) Let U =X + Y and V = X/(X + Y) be two new random variables, and compute a 
probability density ¢ of (U, V). 

(c) Compute a probability density fy . 

(d) Compute a probability density f;. 


. Let X and Y be two independent random variables, each with a standard normal distribution 


(mean = 0, variance = 1). Introduce new random variables U and V by the equations U = 
XIY,V=Y. 
(a) Show that a probability density function of (U, V) is given by the formula 


glu,v) = — = —(4+u0/2 if y <0. 
WT 


(b) Find a similar formula for computing g(u, v) when v > 0. 
(c) Determine a probability density function of U. 


. Assume X has the density function given by 


1 
fx = rJ1 — x2 if -—-l<x<l, 


0 if |x| 21. 
If an independent random variable Y has density 


frQ) - if y20, 
vty) = 


find a density function of Z = XY. 

Given two independent one-dimensional random variables X and Y with continuous densities 
fx and fy. Let Uand V be two random variables such that X = Ucos V, Y = Usin V, with 
U>Oand —-7<V<7r. 

(a) Prove that U has a density such that fy (uw) = 0 for u < 0, and 


0 if y <0, 


fuW = uf fx(ucos v) fy (usin v) do for u2>0. 


(b) Determine fy and the corresponding distribution Fy explicitly when each of X and Y has 
a normal distribution with mean m = 0 and variance o*. 


. (a) Assume o, > 0 and o, > 0. Verify the algebraic identity 


x —m,\? t—x—m,\ x —m,\? t —(m, + m,)° 
eae faa eee ec Sara a Ge ace 
01 D> Oo G 


2 2 

mq + (t — me)oj 
eso +o, => and = ™ =———3 2 
01 + 93 


where 


8. 


9. 
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(b) Given two independent one-dimensional random variables X and Y. Assume X has a 
normal distribution with mean m, and variance a , and that Y has a norma! distribution with 
mean m, and variance o5. Prove that ¥ + Yhas anormal distribution with mean m = m, + mg 
and variance o? = of + o3. 

Given two one-dimensional random variables X and Y with densities fy and fy and joint density 
f. For each fixed y, define 


fx | Y=y)= aa whenever fy(y) > 0. 


This is called the conditional probability density of X, given that Y = y. Similarly, we define 
the conditional probability density of Y, given that XY = x, by the equation 


frY | X=x)= a whenever /fx(x) >0. 
x 


(a) If fy and fy are positive, prove that J", fx(x| Y = y)dx =f" fy(y|X =x) dy =1. 
(b) If fy and fx are positive, prove that 


fx(@) = [° fr Ofc Y=y)dy and f(y) = [° fk@Ofr | X = x) dx. 


A random variable (X, Y) is said to have a normal bivariate distribution if its density function 
fis given by the formula 


yD 
— ~_ 2p Qla,y)/2 
f(y) aa 
where Q(x, y) is the quadratic form 


Q(x, y) = Ay(x — Xo)” + 2Aj.(x — Xo)(y — Yo) + Aooly — yo)". 


The numbers A,,, 412, 4o2 are constants with Aj, >0. The number D = Aj,Ag9 — Aj, is 
called the discriminant of Q and is assumed to be positive. The numbers x, and yg are arbitrary. 
(a) Show that Q(x, y) can be expressed as a sum of squares as follows: 


D 
(x, 9) = An( +) + —v*, where u =X —X9,0 =y — Vo. 
11 


+00 
(b) Define the “improper’’ double integral JJ f(x, y) dx dy to be the limit 


+00 
JJfer dx dy pe) ha dx dy, 


where R(t) is the square [—?t, t] x [—¢, t]. Show that 


[fren dx dy =1. 
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[Hint: Use part (a) to transform the double tntegral over R(t) into a double integral 
in the uv-plane. Then perform a linear change of variables to simplify the integral and, 
finally, lett + +0.] 


10. If a two-dimensional random variable (XY, Y) has a norma! bivariate distribution as described 
in Exercise 9, show that XY and Y themselves are normal one-dimensional random variables 
with means x9 and yg, respectively, and with variances o7(X) = Ag/D, o?(Y) = A,,/D. 

11. If (X, Y) has a normal bivariate distribution as described in Exercise 9, show that the random 
variable Z = X + Y has a one-dimensional normal distribution with mean x, + Yo and 
variance (A,, — 2Aj, + Agy)/D. 


14.25 Expectation and variance 


The mass interpretation of probability distributions may be carried a step further by 
introducing the concepts of expectation and variance. These play the same role in proba- 
bility theory that “‘center of mass’ and “moment of inertia’”’ play in mechanics. Without 
the Stieltjes integral we must give separate definitions for the discrete and continuous cases. 


DEFINITIONS OF EXPECTATION AND VARIANCE. Let X be a one-dimensional randcm variable. 
The expectation of X and the variance of X are real numbers denoted by E(X) and Var (X) 
respectively, and are defined as follows: 

(a) For a continuous random variable with density function fx , 


+00 
E(X) = J tfx(t dt, 


Var (X) =| [t— EXP x(t) at. 


(b) For a discrete random variable with mass points x,, Xz, ... having probabilities 
Px = P(X = x,), we define 


E(X) = xe 
Var (X) = > be — EOP Ps. 


Note: We say that E(X) and Var (X) exist only when the integral or series in question 
is absolutely convergent. It is understood that the series is a finite sum when the sample 
space is finite; in this case E(X) and Var (X) always exist. They also exist when /y is 0 
outside some finite interval. 


The mathematical expectation E(X) is a theoretically computed value associated with 
the random variable X. In some respects, the distribution acts as though its entire mass 
were concentrated at a single point, E(X). The true significance of mathematical expecta- 
tion in probability theory will be discussed in Section 14.29 in connection with the so-called 
“laws of large numbers.”’ 

In mechanics, a knowledge of the center of mass alone gives no indication of how the 
mass is spread or dispersed about its center. A measure of this dispersion is provided by 
the ‘second moment” or “moment of inertia.”’ In probability theory, this second moment 
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is the variance. It measures the tendency of a distribution to spread out from its expected 
value. In Section 14.28 we shall find that a small variance indicates that large deviations 
from the expected value are unlikely. 

Although the expectation E(X) may be positive or negative, the variance Var (X) is 
always nonnegative. The symbol o? is also used to denote the variance. Its positive square 
root is called the standard deviation and is denoted by o. The standard deviation is a 
weighted average; in fact, o is a weighted root mean square of the distance of each value 
of X from the expected value E(X). The analogous concept in mechanics is the “radius of 
gyration.” 


EXAMPLE |. Uniform distribution. Let X have a uniform distribution over an interval 
[a,b]. Then f(t) = 1/(6 — a) ifa<t <b, and f(t) = 0 otherwise. Therefore the expec- 
tation of X is given by 


b? — _a+b 
“22a > 


EX) = | peat —— [ad 


Thus the mean is the mid-point of the interval. If we write m for (a + b)/2 and note that 
m—a=b—m= (b —a)/2 we find 


Var(X) = ["(— my dt = 


— a Ja—m 12 


[out du = Cae 


Note that the variance depends only on the /ength of the interval. 


EXAMPLE 2. Binomial distribution. If X has a binomial distribution with parameters n 


and p we have 
E(X) >. (; ) pra k ae 


k=0 


where g = 1 — p. To evaluate this sum, let 


feat =D (pay 
and note that = 


n 


K(7) xe tytn = ee) y) = n(x at yy" ; 
k=0 x 


If we multiply both sides of this last equation by x and put x = p and y = q, we obtain 
E(X) = np. 
By a similar argument we may deduce the formula 


Var (X) = Dik - no)*(i ) pa kg” * = npq. 


Two proofs of this formula are suggested in Exercise 6 of Section 14.27. 
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EXAMPLE 3. Normal distribution. The terms “‘mean”’ and “‘variance’’ have already been 
introduced in connection with our description of the normal distribution in Section 14.14. 
These terms are justified by the formulas 


BO) = { OY pelle dy = m 
o,/27 —0o 
and 
1 a2 2 
Var (X) = —= { (t — me Ue mloV 2 dy = G?, 
o,/27 —0 


Proofs of these formulas are requested in Exercise 7 of Section 14.27 


Gamblers often use the concept of expectation to decide whether a given game of chance 
is favorable or unfavorable. As an illustration we shall consider the game of betting on 
“red” or “black”’ in roulette. 


EXAMPLE 4. Roulette. A roulette wheel carries the numbers from 0 to 36. The number 
0 appears on a gray background, half of the remaining 36 numbers on a red background, 
and the other half on a black background. The usual methods of betting are: 


(1) Bet $1 on a color (red or black). Possible return: $2. 
(2) Bet $1 on a single number (0 excepted). Possible return: $36. 
(3) Bet $1 on any dozen numbers (0 excepted). Possible return: $3. 


If 0 is the winning number the house wins and all other players lose. 

Let X be the random variable which measures the financial outcome of betting by 
method (1). The possible values of X are x, = —1l and x, = +1. The point probabilities 
are P(X = x,) = 32, P(X = x.) = 44. Therefore the expectation is 


E(X) = (—DF + (+D89 = - 24 


this is usually interpreted to mean that the game is unfavorable to those who play it. The 
mathematical justification for this interpretation is provided by one of the Jaws of large 
numbers, to be discussed in Section 14.29 The reader may verify that the expectation has the 
same value for methods (2) and (3) as well. 


EXAMPLE 5. A coin-tossing game. In a coin-tossing game there is a probability p that 
heads (H) will come up and a probability g that tails (T) will come up, whereO <p < 1 
andq = 1 — p. The goin is tossed repeatedly until the first outcome occurs a second time; 
at this point the game ends. If the first outcome is H we are paid $1 for each T that comes 
up until we get the next H. For example, HTTTH pays $3, but HH pays $0. If the first 
outcome is 7 the same rules apply with H and T interchanged. The problem is to determine 
how much we should pay to play this game. For this purpose we shall consider the random 
variable which counts the number of dollars won and compute its expected value. 

For the sample space we take the collection of all possible games that can be played in 
this manner. This set can be expressed as the union of two sets A and B, where 


A = {TT, THT, THHT, THHHT,...} and B= {HH, HTH, HTTH, HTTTH,...}. 
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We denote the elements of set A (in the order listed) as a), a,, @g, a3, ... and those of set 
B as by, b,, bg, 63, .... Next, we assign the point probabilities as follows: 


P(a,) = p"¢’ and P(b,) = q"p*. 


(When p = 0, we put P(a)) = | and let P(x) = 0 for all other x in A UB. Whensg = 0 
we put P(by) = | and let P(x) = 0 for all other x.) In Section 13.21 1t was shown that this 
is an acceptable assignment of probabilities. 
The random variable X in which we are interested is defined on the sample space A U B 
as follows: 
X(a,) = X(b,) =n for = 071.2. 


The event “X = n’’ consists of the two games a, and b,,, so we have 
P(X =n) = pp"? +q"p’, 


where p® and q° are to be interpreted as 1 when p = 0 org = 0. The expectation of X is 
given by the sum 


(14.40) E(X) = > nP(X =n) = q? > np" + p? > nq". 
n=0 n=0 n=0 


If either p = 0 or g = 0, we obtain E(X) = 0. Otherwise we may compute the sums of 
the series in (14.40) by noting that for 0 < x < 1 we have 


Smensd Send (he 
nx" =x— )x™ =x— = ss 
oo dx “~ dx\1l—x (1 — x) 


Using this in (14.40) with x = p and x = q we obtain, for0 <p <1, 


2 2 
a ergo 2 i 
(l—py (1-4) 
We interpret this result by saying that the game is unfavorable to those who pay more than 
$1 to play it. 

This particular example is of special interest because the expectation E(X) is independent 
of p when 0 < p <1. In other words, loading the coin in favor of heads or tails does not 
affect the expected value except in the extreme cases in which it is so loaded that it always 
falls heads or always falls tails. Note that, as a function of p, the expectation E(X) is 
discontinuous at the points p = 0 and p = 1. Otherwise it has the constant value 1. This 
interesting example was suggested to the author by H. S. Zuckerman. 


E(X) = 


14,26 Expectation of a function of a random variable 


If a new random variable Y is related to a given one X by an equation of the form 
Y = 9(X), its expectation is given (in the continuous case) by the equation 


+0 
(14.41) E(Y) =| tfy(e) dt. 
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The expectation E(Y) can be computed directly in terms of the density fy without deter- 
mining the density of Y. In fact, the following formula is equivalent to (14.41): 


(14.42) EY) = [el fx(s at. 


A proof of (14.42) in the most general case is difficult and will not be attempted here. 
However, for many special cases of importance the proof is simple. In one such case, @ 
is differentiable and strictly increasing on the whole real axis, and takes on every real value. 
For a continuously distributed random variable X with density fy we have the following 
formula for the density function fy (derived in Section 14.17): 


fr) =fxlO): (4), 


where y is the inverse of my. If we use this in (14.41) and make the change of variable 
u = y(t) [so that t = y(u)], we obtain 


+0 +0 + 00 
BY) = [> teat =] fxtyol vO at = J pw fx(w) du, 


which is the same as (14.42). 
When Equation (14.42) is applied to Y = (X — m)?, where m = E(X), we obtain 


E(Y) = [_ (t — m)"fx(0) dt = Var (X). 


This shows that variance is itself an expectation. A formula analogous to (14.42) also 
holds, of course, in the discrete case. More generally, it can be shown that 


Elg(X, Y=] [ox yf (x, y) dx dy 


if (XY, Y) is a continuous random variable with joint density /. 


Note: For two-dimensional random variables, expectation and variance may be 
defined in a manner similar to that used for the one-dimensional case, except that double 
integrals and double sums are employed. We shall not discuss this extension here. 


14.27 Exercises 


1. A die is rolled. Let X denote the number of points on the upturned face. Compute E(X) 
and Var (X). 
2. Assume that X is a continuous random variable with a probability density function. Let 
Y = (X — m)/o, where m = E(X)and o = ./Var (X). Show that E(Y) = Oand E(Y?) = 1. 
3. Derive the following general properties of expectation and variance for either the discrete or 
the continuous case. 
(a) E(cX) = cE(X), where c is a constant. 
(b) Var (cX) = c? Var (X), where c is a constant. 
(c) E(X + Y) = E(X) + E(Y). 
(d) Var (X) = E(X”) — [E(X)P. 
(e) Var (X + Y) = Var (X) + Var (Y) + 2E[(X — E(X))(Y — E(Y))]. 
(f) Ely(X) + 92(Y)] = Ely, (X)] + Elg.(Y)]. [Part (c) is a special case.] 
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4. If X and Y are independent random variables, show that 
(a) Var (X + Y) = Var (X) + Var (Y). 
(b) Ele(X): v(Y)] = Ele(X)]- Ely(Y)]. 
(c) If X,, X2,..., X, are independent random variables with E(X;,) = m;,, show that 


Var > =< m| = > Var (X;, — m;,) = > Var (X;,). 


5. Let X,, X,, X3,..., X, ben independent random variables, each having the same expectation, 
E(X;.) = m, and the same variance, Var (X;,,) = 07. Let X denote the arithmetic mean, X = 
(1/n) >”, X;. Use Exercises 3 and 4 to prove that E(X) = mand Var (X) = o?/n. 

6. (a) Ifg = 1 — p, prove the formula 


$6 -mfiJreso 
k=0 


thereby showing that Var (X) = npg for a random variable X having a binomial distribution 
with parameters n and p. [Hint: k? =k(k — 1) +k.] 
(b) If X has a binomial distribution with parameters n and p, show that X can be expressed 
as a sum of n independent random variables X,, X2,..., X,, each assuming the possible values 
0 and 1 with probabilities p and q, respectively, and each having a binomial distribution. Use 
this result and Exercise 5 to show that E(X) = np and Var (X) = npq. 

7. Determine the expectation and variance (whenever they exist) for a random variable X having 
(a) a Poisson distribution with parameter A. 
(b) a Cauchy distribution. 
(c) an exponential distribution with parameter 4. 
(d) a normal distribution. 

8. A random variable X has a probability density function given by 


Cir), 
[O = Thr if |t|>1, f(t) =0 if |t| <1, 


where r > 1 and C(r) is independent of ¢. 

(a) Express C(r) in terms of r and make a sketch to indicate the nature of the graph of ¢- 

(b) Determine the corresponding distribution function Fy and make a sketch to indicate the 
nature of its graph. 

(c) Compute P(X < 5) and P(5 < X < 10) in terms of r. 

(d) For what values of r does X have a finite expectation? Compute E(X) in terms of r when the 
expectation is finite. 

(e) For what values of r does X have a finite variance? Compute Var (X) in terms of r when 
the variance is finite. 

9. A gambler plays roulette according to the following “‘system.”” He plays in sets of three games. 
In the first and second games he always bets $1 on red. For the third game he proceeds as 
follows: 

(a) If he wins in the first and second games, he doesn’t bet. 

(b) If he wins in one of the first or second and loses in the other, he bets $1 on the color 
opposite to the outcome of the second game. 

(c) If he loses in both the first and second, he bets $3 on red. 

Let X, Y, and Z denote, respectively, the financial outcomes of the first, second, and third 
games. Compute E(X), E(Y), E(Z), and E(X + Y + Z). 
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10. (Petersburg Problem). A player tosses a coin and wins $1 if his first toss is heads. If he tosses 
heads again he wins another dollar. If he succeeds in tossing heads a third time he gets another 
$2 (for a total of $4). As long as he tosses heads in succession x times, his accumulated winnings 
are 2"! dollars. The game terminates when he tosses tails. Let X denote the number of dollars 
won in any particular game. Compute E(X). In view of your result, how much would you 
be willing to pay Harold’s Club in Reno for the privilege of playing this game? 

11. (a) Assume X is a continuous random variable with probability density fy. Let Y= 
(X — m)/o, where m = E(X) and o = /Var (X). Prove that 


E(e¥ ) = e—m/o i et/o Fy(t) dt. 


(b) Let X be a discrete random variable having a Poisson distribution with parameter A. 
Define Y as in part (a) and prove that 


1 = 
E(e¥) = e4Gta) , where G(A) = 1 + Ti ae 


12. A random variable X has a standard normal distribution. Compute: (a) E(|X|), (b) E(e*), 


(c) Var (e*), (d) E(,/ X* + Y*). In part (d), Y also has a standard normal distribution but is 
independent of X. 


14.28 Chebyshev’s inequality 


As mentioned earlier, a small value for the variance means that it is unlikely that a 
random variable X will deviate much from its expected value. To make this statement more 
precise we introduce the absolute value |X — E(X)| which measures the actual distance 
between X and E(X). How likely is it that this distance is more than a given amount? To 
answer this question we must determine the probability 


P[|X — E(X)| > ¢], 
where c is a given positive number. In the continuous case we have 


P[|X — E(X)| > c] = 1 — P[|X — E(X)| Sc] = 1 — P[E(X) -ce SX SK E(X) + ¢] 


= ee d ec d 
ry ae f x(t) dt — eee fx(t) dt 
E(X)—c + 
(14.43) = [oo fe(Qdtt fe tel dt; 


therefore, the calculation of this probability can be accomplished once the density function 
fx isknown. Of course, if fy 1s unknown this method gives no information. However, if the 
variance is known, we can obtain an upper bound for this probability. This upper bound is 
provided by the following theorem of P. L. Chebyshev (1821-1894), a famous Russian 
mathematician who made many important contributions to probability theory and other 
branches of mathematics, especially the theory of numbers. 
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THEOREM 14.11. CHEBYSHEV’S INEQUALITY. Let X be a one-dimensional random variable 
with finite expectation E(X) and variance Var (X). Then for every positive number c we have 


(14.44) PIX — E()| > ce] < 
6 


Proof. In the continuous case we have 


+00 
Var (X) = i [t — E(X)]*fx(t) at 


+ 


E(X)—ce 00 
> ne = BOOP a(o dt + [ft — ECP x(0) at 


(X)+c 


> (ff (t)dt+{— fx(t) dt) 

me NJ x E(X)+e7 * 
Because of (14.43), the coefficient of c? on the right is P[|X — E(X)| > c]. Therefore, when 
we divide by c? we obtain (14.44). This completes the proof for the continuous case; the 
discrete case may be similarly treated. 


Chebyshev’s inequality tells us that the larger we make c the smaller the probability is 
that |X — E(X)| > c. In other words, it is unlikely that X will be very far from E(X); 
it is even more unlikely if the variance Var (X) is small. 

If we replace c by ka, where k > 0 and o denotes the standard deviation, o = y Var (X), 
Chebyshev’s inequality becomes 


PI|X — E(X)| > ko] < > 


That is, the probability that X will differ from its expected value by more than k standard 
deviations does not exceed 1/k?. For example, when k = 10 this inequality tells us that 
the probability P[|X — E(X)| > 100] does not exceed 0.010. In other words, the probability 
is no more than 0.010 that an observed value of X will differ from the expected value by 
more than ten standard deviations. Similarly, when k = 3 we find that the probability 
does not exceed § = 0.111... that an observed value will differ from the mean by more than 
three standard deviations. 

Chebyshev’s inequality is a general theorem that applies to all distributions. In many 
applications the inequality can be strengthened when more information is known about 
the particular distribution. For example, if X has a binomial distribution with parameters 
n and p it can be shown (by use of the normal approximation to the binomial distribution) 
that for large n the probability is about 0.003 that an observed value will differ from the 
mean by more than three standard deviations. (For this result, n > 12 suffices.) This is 
much smaller than the probability 0.111 provided by Chebyshev’s inequality. 


EXAMPLE. Testing a coin for fairness. We want to decide whether or not a particular 
coin 1s fair by tossing it 10,000 times and recording the number of heads. For a fair coin 
the random variable Y which counts the number of heads has a binomial distribution with 
parameters n = 10,000 and p= %. The mean of X is np = 5,000 and the standard 
deviation is o = ./npqg = 50. (See Example 2 in Section 14.25.) As mentioned above, the 
probability for a binomially distributed random variable to differ from its expected value 
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by more than 3o is about 0.003. Therefore, let us agree to say that a coin is not fair if 
the number of heads in 10,000 tosses differs from the mean by more than 30. Since 
E(X) = 5,000 and 36 = 150, we would say the coin is unfair if the number of heads in 
10,000 tosses is less than 4,850 or more than 5,150. 


14.29 Laws of large numbers 


In connection with coin-tossing problems, it is often said that the probability of tossing 
heads with a perfectly balanced coin is 3. This does not mean that if a coin is tossed twice 
it will necessarily come up heads exactly once. Nor does it mean that in 1000 tosses heads 
will appear exactly 500 times. Let us denote by A(m) the number of heads that occur in n 
tosses. Experience shows that even for very large n, the ratio h(n)/n is not necessarily $. 
However, experience also shows that this ratio does seem to approach } as n increases, 
although it may oscillate considerably above and below 3 in the process. This suggests that 
it might be possible to prove that 


(14.45) roel eo 


Unfortunately, this cannot be done. One difficulty is that the number h(n) depends not 
only on n but also on the particular experiment being performed. We have no way of 
knowing in advance how h(n) will vary from one experiment to another. But the real 
trouble is that it is possible (although not very likely) that in some particular experiment 
the ratio h(n)/n may not tend to 3 at all. For example, there is no reason to exclude the 
possibility of getting heads on every toss of the coin, in which case h(n) = nand h(n)/n > 1. 
Therefore, instead of trying to prove the formula in (14.45), we shall find it more reasonable 
(and more profitable) to ask how likely it is that A(n)/n will differ from 4 by a certain 
amount. In other words, given some positive number c, we seek the probability 


p((™P-3)>¢). 


2 
By introducing a suitable random variable and using Chebyshev’s inequality we can get 
a useful upper bound to this probability, a bound which does not require an explicit 
knowledge of h(n). This leads to a new limit relation that serves as an appropriate substitute 
for (14.45). 

No extra effort is required to treat the more general case of a Bernoullian sequence of 
trials, in which the probability of “success”? is p and the probability of “‘failure’’ is q. 
(In coin tossing, ‘“‘success’’ can mean “‘heads’’ and for p we may take 4.) Let X denote the 
random variable which counts the number of successes in n independent trials. Then XY 
has a binomial distribution with expectation E(X) = np and variance Var (X) = npq. 
Hence Chebyshev’s inequality is applicable; it states that 


(14.46) P(\X — np| >c) <=. 
Cc 


Since we are interested in the ratio X/n, which we may call the relative frequency of success, 
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we divide the inequality |X — np| > c by n and rewrite (14.46) as 
(14.47) p(2 -2|><) < "Bt 
n n c 


Since this is valid for every c > 0, we may let c depend on n and write c = en, where 
€ is a fixed positive number. Then (14.47) becomes 


e 


r(Z-of>9 
n n 
The appearance of n in the denominator on the right suggests that we let n > 00. This 
leads to the limit formula 


(14.48) lim P(| ee p | > ‘ = 0 for every fixed «€>0, 
n> co n 

called the law of large numbers for the Bernoulli distribution. It tells us that, given any 

e« > 0 (no matter how small), the probability that the relative frequency of success differs 

from p by more than e is a function of n which tends to 0 asn—> oo. This limit relation 

gives a mathematical justification to the assignment of the probability 4 for tossing heads 

with a perfectly balanced coin. 

The limit relation in (14.48) 1s a special case of a more general result in which the 
“relative frequency’ X/n is replaced by the arithmetic mean of n independent random 
variables having the same expectation and variance. This more general theorem is usually 
referred to as the weak law of large numbers; it may be stated as follows: 


THEOREM 14.12. WEAK LAW OF LARGE NUMBERS. Let X,, X2,..., X, ben independent 
random variables, each having the same expectation and the same variance, say 


E(X;,) = m and Var (X,) =o? for k=1,2,...,n. 


Define a new random variable X (called the arithmetic mean of X,, Xz, ..., X,) by the 


equation 
_ wr 
LS) hy 
n — 
Then, for every fixed « > 0, we have 


(14.49) lim P(|X — m| > «) = 0. 
An equivalent statement is 


(14.50) lim P(|¥ — ml <= 1. 


n> 
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Proof. We apply Chebyshev’s inequality to X. For this we need to know the expectation 
and variance of X¥. These are 


2 


E(Y)=m and Var(X)=—. 
n 


(See Exercise 5 in Section 14.27.) Chebyshev’s inequality becomes P(|X — m| > c) < 
o*/(nc?). Letting n — oo and replacing c by e we obtain (14.49) and hence (14.50). 


Note: To show that the limit relation in (14.48) is a special case of Theorem 14.12, we 
assume each X;, has the possible values 0 and 1, with probabilities P(Y, = 1) = p and 
P(X, =0) =1—p. Then X is the relative frequency of success in n independent trials, 
E(X ) = p, and (14.49) reduces to (14.48). 


Theorem 14.12 is called a weak law because there is also a strong law of large numbers 
which (under the same hypotheses) states that 
(14.51) P (tim beer ee 0} ie 
The principal difference between (14.51) and (14.50) is that the operations “limit”? and 
“probability” are interchanged. It can be shown that the strong law implies the weak law, 
but not conversely. 

Notice that the strong law in (14.51) seems to be closer to formula (14.45) than (14.50) is. 
In fact, (14.51) says that we have lim, _.,, X = m “‘almost always,” that is, with probability 
1. When applied to coin tossing, in particular, it says that the failure of Equation (14.45) 
is no more likely than the chance of tossing a fair coin repeatedly and always getting heads. 
The strong law really shows why probability theory corresponds to experience and to our 
intuitive feeling of what probability “should be.” 

The proof of the strong law is lengthy and will be omitted. Proofs appear in the books 
listed as References 1, 3, 8, and 10 at the end of this chapter. 


14.30 The central limit theorem of the calculus of probabilities 


In many applications of probability theory, the random variables of interest are sums of 
other random variables. For example, the financial outcome after several plays of a game 
is the sum of the winnings at each play. A surprising thing happens when a large number 
of independent random variables are added together. Under general conditions (applicable 
in almost every situation that occurs in practice) the distribution of the sum tends to be 
normal, regardless of the distributions of the individual random variables that make up the 
sum. The precise statement of this remarkable fact is known as the central limit theorem of 
the calculus of probabilities. \t accounts for the importance of the normal distribution 
in both theory and practice. A thorough discussion of this theorem belongs to the 
advanced study of probability theory. This section will merely describe what the theorem 
asserts. 

Suppose we have an infinite sequence of random variables, say X,, X2,..., with finite 
expectations and variances. Let 


m,= E(X,) and of =Var(X,), k=1,2,.... 
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We form a new random variable S, by adding the first n differences Xj, — m,: 


(14.52) S, =>(X, — m,). 
k=1 


We add the differences rather than the X, alone so that the sum S, will have expected 
value 0. The problem here is to determine the limiting form, as n — oo, of the distribution 
function of S,,. 

If X,, X2,..., X,, are independent, then [by Exercise 4(c) of Section 14.27] we have 


Var (S,,) = > Var (X;, — m,) = > Var (X,) = >of. 
k=1 k=1 k=1 


Ordinarily, Var (S,,) will be large even though the individual variances of may be small. 
Random variables with a large variance are not fruitful objects of study because their 
values tend to be widely dispersed from the expected value. For this reason, a new random 
variable 7, is introduced by the equation 


Sn 


Var (S,) 
This new variable has expectation 0 and variance 1 and is called a standardized random 
variable. The standardized variable 7, is meaningful even if the random variables X,, 
X,,..., X, are not independent. 

We now introduce the following definition: 


DEFINITION OF THE CENTRAL LIMIT PROPERTY. Let 
(14.54) Se oem nee 


be a sequence of random variables (not necessarily independent), where each X,, has a finite 
expectation m, and a finite variance o%. Define S, and T,, by (14.52) and (14.53). The 
sequence in (14.54) is said to satisfy the central limit property if, for alla and b witha < b, 
we have 


b 2 
(14.55) lim P(a < T, <b) = = en dy, 


n> oO a a 


In other words, the random variables in (14.54) satisfy the central limit property if the 
distribution of the standardized variable 7,, approaches a standard normal distribution 
asn— > oo. [Equation (14.55) is to hold also ifa = —o orb =+0.] 

Laplace was the first to realize that this property is shared by many sequences of random 
variables, although a special case (random variables describing a Bernoullian sequence of 
trials) had been known earlier by DeMoivre. (Figure 14.11 shows a binomial distribution 
and a corresponding normal approximation.) Laplace stated a general central limit 
theorem which was first completely proved by the Russian mathematician A. Lyapunov in 
1901. In 1922, J. W. Lindeberg generalized Laplace’s result by showing that the property 


568 Calculus of probabilities 


is satisfied if the random variables are independent and have a common distribution giving 
them the same expectations and variances, say E(X,) = m and Var (X;,) = o for all k. 
In this case the standardized variable becomes 


Sake Thm 


on 


Lindeberg realized that independence alone is not sufficient to guarantee the central limit 
property, but he formulated another condition (now known as the Lindeberg condition) 
which, along with independence, is sufficient. In 1935, W. Feller showed that the Lindeberg 
condition is both necessary and sufficient for independent random variables to satisfy the 
central limit property. We shall not discuss the Lindeberg condition here except to mention 
that it implies 


T, = 


Var (S,,) — © as n—-oo. 


Fortunately, many independent random variables that occur in practice automatically 
satisfy the Lindeberg condition and therefore also have the central limit property. Up 
to now, the theory for dependent random variables is incomplete. Only a few special 
cases have been treated. Much of the contemporary research in probability theory centers 
about the search for general theorems dealing with dependent variables. 


14.31 Exercises 


1. Carry out the proof of Chebyshev’s inequality in the discrete case. 
2. If a is any real number, prove that 


1 
P(|X — al > cA) $3 


for every c > 0, where A? = [+% (t — a)*fx(t) dt. Chebyshev’s inequality is the special case 
in which a = E(X). 

3. Let X denote the random variable which counts the number of successes in n independent 
trials of a Bernoullian sequence; the probability of success is p. Show that, for every « > 0, 


P ca a 
Pia 4 edi ea TC 


4. A fair coin is tossed n times; the number of heads is denoted by X. Find the smallest n for 
which Chebyshev’s inequality implies 


X 
p(o4 < = < 06) > 0.90. 


5. In a production line the number X of the defective articles manufactured in any given hour is 
known to have a Poisson distribution with mean E(X) = 100. Use Chebyshev’s inequality 
to compute a lower bound for the probability that in a given hour there will be between 90 and 
110 defective articles produced. 

6. Assume that a random variable X has a standard norma! distribution (mean 0 and variance 
1). Let p denote the probability that X differs from its expectation E(X) by more than three 
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times its standard deviation. Use Chebyshev’s inequality to find an upper bound for p. Then 
use Suitable tables of the normal distribution to show that there is an upper bound for p that 
is approximately one-fiftieth of that obtained by Chebyshev’s inequality. 


. Given a sequence of independent random variables X,, X2,..., each of which has a normal 


distribution. Let m, = E(X;,) and let o2 = Var (X,). Show that this sequence has the central 
limit property. [Hint: Refer to Exercise 7 in Section 14.24.] 


. Let X,, X2,... be independent random variables having the same binomial distribution. 


Assume each X, takes the possible values 0 and 1 with probabilities PLY, = 1) =p and 
P(X, = 0) =q, wherep +q =1. LetZ, = X, +-:- + X,. Therandom variable Z,, counts 
the number of successes in n Bernoulli trials. 

(a) Show that the central limit property takes the following form: 


t 
im P( == ”P <:) -—|- en wl2 dy, 
n—>00 s/npq J/20 = 


(b) Use the approximation suggested by part (a) to estimate the probability of obtaining 
between 45 and 55 heads if a fair coin is tossed 100 times. Refer to Table 14.1, p. 536 for the 
computation. 


. With the notation of Exercise 8, the central limit theorem for random variables describing a 


Bernoullian sequence of trials can be written in the form 


(ns ar “ae st 
lim “4 = 


n—> 0 O(7,) =a O(t,) = 


where ® is the standard normal distribution. For this particular case it can be shown that the 
formula is also valid when f, and f, are functions of n given by t, = (a — np)/ a npg and tz = 


(b — np)/ / npg, where a and b are fixed positive constants, a < b. 
(a) Show that this relation implies the asymptotic formula 


a _ ee | 
> (7) ar ~ o( 2 mp +2 ") - o(-—2—*) as n— oo, 
on /npq /npq 


(b) An unbiased die is tossed 180 times. Use the approximation suggested in part (a) to esti- 
mate the probability that the upturned face is a six exactly 30 times. Refer to Table 14.1, 
p. 536 for the computation. 


. An unbiased die is tossed 100 times. Use the approximation suggested in Exercise 9(a) to 


estimate the probability that the upturned face is a six (a) exactly 25 times, (b) at least 25 
times. Refer to Table 14.1, p. 536 for the computation. 
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INTRODUCTION TO NUMERICAL ANALYSIS 


15.1 Historical introduction 


The planet Uranus was discovered in 1781 by a gifted amateur astronomer, William 
Herschel (1738-1822), with a homemade 10-ft. telescope. With the use of Kepler’s laws, 
the expected orbit of Uranus was quickly calculated from a few widely separated observa- 
tions. It was found that the mean distance of Uranus from the sun was about twice that 
of Saturn and that one complete orbit would require 84 years. By 1830 the accumulated 
empirical data showed deviations from the scheduled orbit that could not be accounted 
for. Some astronomers felt that Newton’s law of universal gravitation might not hold for 
distances as large as that of Uranus from the sun; others suspected that the perturbations 
were due to a hitherto undiscovered comet or more distant planet. 

An undergraduate student at Cambridge University, John Couch Adams (1819-1892), 
was intrigued by the possibility of an undiscovered planet. He set himself the difficult 
task of calculating what the orbit of such a planet must be to account for the observed 
positions of Uranus, assuming the validity of Newton’s law of gravitation. He completed 
his calculations in 1845 and asked the Royal Observatory at Greenwich to search for the 
hypothetical planet, but his request was not taken seriously. 

A similar calculation was made independently and almost simultaneously by Jean 
Joseph Leverrier (1811-1877) of Paris, who asked Johann Galle, head of the Berlin 
Observatory, to confirm his prediction. The same evening that he received Leverrier’s 
letter, Galle found the new planet, Neptune, almost exactly in its calculated position. 
This was another triumph for Newton’s law of gravitation, and one of the first major 
triumphs of numerical analysis, the art and science of computation. 

The history of numerical analysis goes back to ancient times. As early as 2000 B.C. 
the Babylonians were compiling mathematical tables. One clay tablet has been found 
containing the squares of the integers from 1 to 60. The Babylonians worshipped the 
heavenly bodies and kept elaborate astronomical records. The celebrated Alexandrian 
astronomer Claudius Ptolemy (circa 150 A.D.) possessed a Babylonian record of eclipses 
dating from 747 B.C. 

In 220 B.C., Archimedes used regular polygons as approximations to a circle and 
deduced the inequalities 33? < a < 33. Numerical work from that time until the 17th 
century was centered principally around the preparation of astronomical tables. The 
advent of algebra in the 16th century brought about renewed activity in all branches of 
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mathematics, including numerical analysis. In 1614, Napier published the first table of 
logarithms. In 1620, the logarithms of the sine and tangent functions were tabulated to 
seven decimal places. By 1628, fourteen-place tables of the logarithms of the numbers 
from 1 to 100,000 had been computed. 

Computations with infinite series began to flourish near the end of the 17th century, 
along with the development of the calculus. Early in the 18th century Jacob Stirling and 
Brook Taylor laid the foundations of the calculus of finite differences, which now plays a 
central role in numerical analysis. With the prediction of the existence and location of 
the planet Neptune by Adams and Leverrier in 1845, the scientific importance of numerical 
analysis became established once and for all. 

Late in the 19th century the development of automatic calculating machinery further 
stimulated the growth of numerical analysis. This growth has been explosive since the 
end of World War II because of the progress in high-speed electronic computing devices. 
The new machines have made possible a great many outstanding scientific achievements 
which previously seemed unattainable. 

The art of computation (as distinct from the science of computation) lays much stress 
on the detailed planning required in a particular calculation. It also deals with such 
matters as precision, accuracy, errors, and checking. This aspect of numerical analysis 
will not be discussed here; it is best learned by carrying out actual numerical calculations 
with specific problems. For valuable advice on practical methods and techniques the 
reader should consult the existing books on numerical analysis, some of which are listed 
in the bibliography at the end of this chapter. The bibliography also contains some of the 
standard mathematical tables; many of them also give practical information on how to 
carry Out a specific calculation. 

This chapter provides an introduction to the science of computation. It contains some 
of the basic mathematical principles that might be required of almost anyone who uses 
numerical analysis, whether he works with a desk calculator or with a large-scale high- 
speed computing machine. Aside from its practical value, the material in this chapter is 
of interest in its own right, and it is hoped that this brief introduction will stimulate the 
reader to learn more about this important and fascinating branch of mathematics. 


15.2 Approximations by polynomials 


A basic idea in numerical analysis is that of using simple functions, usually polynomials, 
to approximate a given function f. One type of polynomial approximation was discussed 
in Volume I in connection with Taylor’s formula (Theorem 7.1). The problem there was 
to find a polynomial P which agrees with a given function f and some of its derivatives 
at a given point. We proved that if fis a function with a derivative of order n at a point 
a, there is one and only one polynomial P of degree <n which satisfies the n + 1 relations 


Pa)=f(@), Pla=f'@, ..., Pa) =f™a). 


The solution is given by the Taylor polynomial, 
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We also discussed the error incurred in approximating f(x) by P(x) at points x other than a. 
This error is defined to be the difference E,,(x) = f(x) — P(x), so we can write 


nm. ¢k) 
fx) => x — a + £00). 


k=0 


To make further statements about the error we need more information about f. For 
example, if fhas a continuous derivative of order n + 1 in some interval containing a, then 
for every x in this interval the error can be expressed as an integral or as an (n + l1)st 
derivative: 


x 7 _— 4\re (ntl) me a _— 7\nrtl 
Ew) == |G nse di = LA — a 


where c lies between a and x. (See Sections 7.5 and 7.7 in Volume I.) 

There are many other ways to approximate a given function f by polynomials, depending 
on the use to be made of the approximation. For example, instead of asking for a poly- 
nomial that agrees with f and some of its derivatives at a given point, we can ask for a 
polynomial that takes the same values as f at a number of distinct points. Specifically, if 


the given distinct points are x9, x;, ..., X, we seek a polynomial P satisfying the 
conditions 
(15.1) P(X) =f(%o), PO) =fOa), wee POn) = fOn)- 


Since there are n + | conditions to be satisfied we try a polynomial of degree <n, say 
P(x) = > a,x", 
k=0 


with n + | coefficients a), a,,..., a, to be determined. Then + 1 conditions (15.1) lead 
to a system of n + 1 linear equations for the coefficients. From the theory of linear equa- 
tions it can be shown that this system has one and only one solution; hence such a poly- 
nomial always exists. If the equations are solved by Cramer’s rule the coefficients ap, 
Q,,..., 4, are expressed as quotients of determinants. In practice, however, the poly- 
nomial P is seldom determined in this manner because the calculations are extremely 
laborious when n is large. Simpler methods have been developed to calculate the poly- 
nomial approximation. Some of these will be discussed in later sections. The polynomial 
which solves the foregoing problem is called an interpolating polynomial. 

Another common type of polynomial approximation is the so-called Jeast-square 
approximation. Here the given function f is defined and integrable on an interval [a, 5] 
and we seek a polynomial P of degree <n such that the mean-square error 


[? fe) — POOP ax 


will be as small as possible. In Section 15.4 we shall prove that for a continuous f such a 
polynomial exists and is uniquely determined. The Legendre polynomials introduced in 
Section 1.14 play a fundamental role in the solution of this problem. 
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15.3 Polynomial approximation and normed linear spaces 


All the different types of polynomial approximation described in the foregoing section 
can be related by one central idea which is best described in the language of linear spaces. 

Let V be a linear space of functions which contains all polynomials of degree <n and 
which also contains the function f to be approximated. The polynomials form a finite- 
dimensional subspace S, with dim S = n+ 1. When we speak of approximating f by a 
polynomial P in S, we consider the difference f — P, which we call the error of the approxi- 
mation, and then we decide on a way to measure the size of this error. 

If V is a Euclidean space, then it has an inner product (x, y) and a corresponding norm 
given by ||x|| = (x, x)’*, and we can use the norm | f — P|| as a measure of the size of 
the error. 

Sometimes norms can be introduced in non-Euclidean linear spaces, that is, in linear 
spaces which do not have an inner product. These norms were introduced in Section 7.26. 
For convenience we repeat the definition here. 


DEFINITION OF A NORM. Let V be a linear space. A real-valued function N defined on V 
is called a norm if it has the following properties: 

(a) N(f) > 0 for all f in V. 

(b) M(cf) = |c| Nf) for all fin V and every scalar c. 

(c) Nif+g)< NY) + Mg) for all f and g in V. 

(d) Nf) = 0 implies f= O. 


A linear space with a norm assigned to it is called a normed linear space. 

The norm of fis sometimes written ||f|| instead of N(f). In this notation, the funda- 
mental properties become: 

(a) fll > 0, 

(b) lief =Iel If, 

(c) If tell < fll + lel. 

(d) || || = 0 implies f= 0. 

A function N that satisfies properties (a), (b), and (c), but not (d), is called a seminorm. 
Some problems in the theory of approximation deal with seminormed linear spaces; 
others with normed linear spaces. The following examples will be discussed in this chapter. 


EXAMPLE |. Taylor seminorm. For.a fixed integer n > 1, let V denote the linear space of 
functions having a derivative of order n at a given point a. If fe V, let 


NA) =S1F*(a)I. 


It is easy to verify that the function N so defined is a seminorm. It is not a norm because 
N(f) = 0 if and only if 
f@ =f'@ =---=f™a) =0, 


and these equations can be satisfied by a nonzero function. For example, N(f) = 0 when 


f(x) = (x -— ay”. 
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EXAMPLE 2. Interpolation seminorm. Let V denote the linear space of all real-valued 
functions defined on an interval [a,b]. For a fixed set of n+ 1 distinct points xp, 
X1,...,X, in [a, b], let N be defined by the equation 


Nf) = SIs) 


if fe V. This function N is a seminorm on V. It is not a norm because N(/) = 0 if and 
only if f(x) = f(%) =-++: =f(x,) = 0, and it is clear that these equations can be satis- 
fied by a function f that is not zero everywhere on [a, 5]. 


EXAMPLE 3. Square norm. Let C denote the linear space of functions continuous on 
an interval [a, b]. If fe C define 


(15.2) Nit) = ([P LFGOR ax)”. 


This is a norm inherited from the inner product 
b <a 
(f, g) = if f (x)g(x) dx. 


Note: Let S denote the set of functions f that are integrable on [a, b]. The set Sis a 
linear space, and the function N defined by (15.2) is a seminorm on S. It is not a norm 
because we can have N(/f) = 0 without f being identically zero on [a, 5}. 


EXAMPLE 4. Max norm. Let C denote the linear space of functions continuous on an 
interval [a, 5). If fe C, define 


N(f) = max |f(2)], 


where the symbol on the right stands for the absolute maximum value of | f| on [a, b]. The 
verification of all four norm properties is requested in Exercise 4 of Section 15.5. 


15.4 Fundamental problems in polynomial approximation 


Let C be the space of functions continuous on a given interval [a, 5], and let S be the 
linear subspace consisting of all polynomials of degree <n. Assume also that a norm or 
seminorm has been defined on C. Choose a function fin C. If there is a polynomial P in S 
such that 


If- PI < If- Ql 


for all polynomials Q in S, we say that P is a best polynomial approximation to f with the 
specified degree. The term “‘best’’ 1s, of course, relative to the given norm (or seminorm). 
The best polynomial for one choice of norm need not be best for another choice of norm. 

Once a norm or seminorm has been chosen, three problems immediately suggest them- 
selves. 


1. Existence. Given f in C, is there a best polynomial approximation to f with the 
specified degree? 
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2. Uniqueness. If a best polynomial approximation to f exists with the specified degree, 
is it uniquely determined? 


3. Construction. If a best polynomial approximation to fexists with the specified degree, 
how can it be determined? 


There are, of course, many other problems that can be considered. For example, if a 
unique best polynomial P,, of degree <n exists, we may wish to obtain upper bounds for 
[f — P,,|| that can be used to satisfy practical requirements. Or we may ask whether 
if — P,, || -~ 0 as n — oo for the given norm or possibly for some other norm. If so, we 
say that the polynomial approximations converge to f in this norm. In such a case 
arbitrarily close approximations exist relative to this norm if n is sufficiently large. These 
examples illustrate some of the types of problems considered in the general theory of 
polynomial approximation. In this introductory treatment we restrict our attention 
primarily to the three problems of existence, uniqueness, and construction, as described 
above. 

For approximation by Taylor polynomials these three problems can be completely 
solved. If f has a derivative of order n at a point a, it is easy to prove that the best poly- 
nomial approximation of degree <n relative to the Taylor seminorm for this 7 is the 
Taylor polynomial 


(15.3) P(x) -SLO« —a)*. 


In fact, for this polynomial we have 


If — PL=SIS™@ — Pa) = 0, 


so the inequality || f— P|| < || — Q|| is trivially satisfied for all polynomials Q. Therefore 
P is a best polynomial approximation relative to this seminorm. To establish uniqueness, 
we consider any polynomial Q of degree <n such that || f— Q|| =0. This equation 
implies that 


O@a=fa@, AW@=f@, «.--, WaO=f@. 


From Theorem 7.1 of Volume I we know that the Taylor polynomial in (15.3) is the only 
polynomial satisfying all these equations. Therefore Q = P. Equation (15.3) also solves 
the problem of construction. 

All three problems can also be solved for any norm derived from an inner product. In 
this case, Theorem 1.16 tells us that there is a unique polynomial in S for which the norm 
|| f — P| is as small as possible. In fact, this P is the projection of fon S and is given by 


an explicit formula, 
n 


P(x) = > (P, ee(x), 


k=0 


where @), €1,..., @, are functions forming an orthonormal basis for S. 
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For example, if C is the space of real functions continuous on the interval [—1, 1] and if 
1 
(f, 2) =] f(x)g(x) dx, 


the normalized Legendre polynomials 9g), 9,,..., 9, form an orthonormal basis for S, 
and the projection f,, of fon S is given by 


Ful) =SF pe ordx), where (fw) = | fOa(0) dt. 


We recall that the normalized Legendre polynomials are given by 


2k+1 1 d* 
V(x) = f a P,(x), where P,(x) = ares 7” (x? — 1)*. 


The first six normalized polynomials are 


g(x) =V, glx) =V3x, a(x) = AVE GBx?- 1), (x) = V3 (5x? — 3), 


4(x) = 4V2 (35x4 — 30x2 +3), —-@3(x) = AV 42 (63x — 70x3 + 15x). 


The corresponding problems for the interpolation seminorm will be treated next in 
Section 15.6. In later sections we discuss polynomial approximation relative to the max 
norm. 


15.5 Exercises 


1. Prove that each of the following collections of functions is a linear space. 
(a) All polynomials. 
(b) All polynomials of degree <n. 
(c) All functions continuous on an interval J. 
(d) All functions having a derivative at each point of J. 
(e) All functions having a derivative of order n at each point of J. 
(f) All functions having a derivative of order n at a fixed point xp. 
(g) All functions having power-series expansions in a neighborhood of a given point x. 
2. Determine whether or not each of the following collections of real-valued functions is a linear 
space. 
(a) All polynomials of degree n. 
(b) All functions defined and bounded on an interval [a, 5]. 
(c) All step functions defined on an interval [a, 5]. 
(d) All functions monotonic on an interval [a, 5]. 
(e) All functions integrable on an interval [a, 5]. 
(f) All functions that are piecewise monotonic on an interval [a, 5]. 
(g) All functions that can be expressed in the form f — g, where f and g are monotonic in- 
creasing on an interval [a, 5]. 
3. Let C denote the linear space of real-valued functions continuous on an interval [a, 6]. A 
function N is defined on C by the equation given. In each case, determine which of the four 
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properties of a norm are satisfied by N, and determine thereby whether N is anorm, a seminorm, 
or neither. 


(a) M(f) =f@). ©) Ni) =| [?f@ ax]. 
(b) Nf) =|f@). (f) NC) = |. If@dl de. 
©) Nf) =If® - fal. (g) Nf) = |’ If@P dx. 
(@) Nf) = [° fax. ny NN =| [’F@) dx)", 


. Let C be the linear space of functions continuous on an interval [a, b]. If fe C, define 


N(f) = ay 


Show that N is a norm for C. 


. Let B denote the linear space of all real-valued functions that are defined and bounded on an 


interval [a, b]. If fe B, define 
N(f) = sup |f(@|, 
a<a<b 


where the symbol on the right stands for the supremum (least upper bound) of the set of all 
numbers | f(x)| for x in [a, 6]. Show that N is a norm for B. This is called the sup norm. 


. Refer to Exercise 3. Determine which of the given functions N have the property that 


N(fg) < N(f)N(g) for all fand g in C. 


. For a fixed integer n > 1, let S be the set of all functions having a derivative of order n at a 


fixed point x). If fe S, let 
N(f) = > J (r) 
(N= > Gyan. 
k=0 
(a) Show that N is a seminorm on S. 
(b) Show that N( fg) < N(/P)N(g) for all f, gin S. Prove also that the Taylor seminorm does 
not have this property. 


. Let f be a real continuous function on the interval [—1, 1]. 


(a) Prove that the best quadratic polynomial approximation relative to the square norm on 
[—1, 1] is given by 


P(x) = 4" fadat + 3x [" af(adt + §Gx2 —1) |] GA - DfOae. 


(b) Find a similar formula for the best polynomial approximation of degree <4. 


. Calculate constants a, b,c so that the integral J*, |e” — (a + bx + cx*)|? dx will be as small as 


possible. 
Let f(x) = |x| for -1 < x <1. Determine the polynomial of degree <4 that best approxi- 


mates fon [—1, 1] relative to the square norm. 


. Let C denote the linear space of real continuous functions on [a, 6] with inner product 


(f.g) = JS? f(x)g(x) dx. Let eg,..., e, be an orthonormal basis for the subspace S of poly- 
nomials of degree <n. Let P be the polynomial in S that best approximates a given fin C 
relative to the square norm. 

(a) Prove that the square of the norm of the error is given by 


lf — Pll? = If? — 26 ex). 


(b) Calculate this error explicitly when [a, b] = [-1, 1], n =2, and f(x) = |x|. 
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12. Let f(x) = 1/x for x #0. 
(a) Show that the constant polynomial P that best approximates f over the interval [1, 7] 
relative to the square norm is P(x) = (log n)/(n — 1). Compute ||P — f|||? for this P. 
(b) Find the linear polynomial P that best approximates f over the interval [1, 7] relative to 
the square norm. Compute ||P — f||? for this P when n = 2. 

13. Let f(x) =e". 
(a) Show that the constant polynomial P that best approximates f over the interval [0, 7] 
relative to the square norm is P(x) = (e" — 1)/n. Compute ||P — f ||? for this P. 
(b) Find the linear polynomial P that best approximates f over the interval [0, 1] relative to 
the square norm. Compute ||P — f ||? for this P. 

14. Let Py, Py,...,P, ben +1 polynomials orthonormal on [a, 5] relative to the inner product 
in Exercise 11. Assume also that P;, has degree k. 
(a) Prove that any three consecutive polynomials in this set are connected by a recurrence 
relation of the form 

Prei4(%) = (Ayx + by)P(X) + C4Pp_iCx) 


for! <k <n — 1, where a,, b,, c, are constants. 
(b) Determine this recurrence relation explicitly when the polynomials are the orthonormal 
Legendre polynomials. 
15. Refer to Exercise 14, and let p;, denote the coefficient of x* in P;,(x). 
(a) Show that a, = pyis/Px- 
(b) Use the recurrence relation in Exercise 14 to derive the formula 


Pm Pins t(x)Pm(y) 7 Pin(X)PmiiQ) 
Pm+1 ey 


b 


> PCP.) = 
k=0 


valid for x # y. Discuss also the limiting case x = y. 


15.6 Interpolating polynomials 


We turn now to approximation by interpolation polynomials. The values of a function 
fare known at n + | distinct points x9, x,,..., X, and we seek a polynomial P of degree 
<n that satisfies the conditions 


(15.4) P(Xo) =f(%), POY =fCa), ees Pn) = f%n)- 


First we prove that if such a polynomial exists it is unique. Then we prove it exists by 
explicit construction. This polynomial minimizes the distance from f to P, measured in 
the interpolation seminorm for this 7, 


If — Pll = SIs) — P(x,)]. 


Since this distance is 0 if P satisfies (15.4), the interpolating polynomial P is the best 
approximation relative to this seminorm. 


THEOREM 15.1. UNIQUENESS THEOREM. Given n + 1 distinct points x9, X1,...5, Xn, let 
P and Q be two polynomials of degree <n such that 


P(x,) = Q(%;) 
for eachk =0,1,2,...,. Then P(x) = Q(x) for all x. 
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Proof. Let R(x) = P(x) — Q(x). The function R is a polynomial of degree <n which 
has n + 1 distinct zeros at the points x9, x,, ..., X,. The only polynomial with this 
property is the zero polynomial. Therefore R(x) = 0 for all x, so P(x) = Q(x) for all x. 


The interpolating polynomial P can be constructed in many ways. We describe first a 
method of Lagrange. Let A(x) be the polynomial given by the equation 


(15.5) A(x) = (x — xp)(e — x4) + (8 — x,) = Te — x). 


This polynomial has a simple zero at each of the points x;. Let A,(x) denote the poly- 
nomial of degree n obtained from A(x) by deleting the factor x — x,. That is, let 


(15.6) A,(x) = TT —x,). 


The polynomial A,(x) has a simple zero at each point x; # x,. At the point x, itself we 
have 


(15.7) A,(X;) = TT Ge — x,). 


j=0 


This is nonzero since no factor in the product is zero. Therefore the polynomial 
A,(x)/A;,(x;,) has the value 1 when x = x, and the value 0 when x = x, for x; # x,. Now 
let 


K=0  Ag(Xq) 


When x = x;, each term in this sum vanishes except the jth term, which has the value 
J (x;). Therefore P(x;) = f(x,;) for each j. Since each term of this sum is a polynomial of 
degree n, the sum itself is a polynomial of degree <n. Thus, we have found a polynomial 
satisfying the required conditions. These results can be summarized by the following 
theorem: 


THEOREM 15.2. Given n+ 1 distinct points xX), x1, ..., X, and n+ 1 real numbers 
FS (Xo), f(%1)> «+ + f(Xn), not necessarily distinct, there exists one and only one polynomial P of 
degree <n such that P(x;) = f(x,) for each] = 0,1,2,...,. This polynomial is given by 
the formula 


(15.8) P(x) = SS (x4) An) 


k=0 A,(x ») 


9 


where A,(x) is the polynomial defined by (15.6). 
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Formula (15.8) for P(x) is called Lagrange’s interpolation formula. We can write it in 
the form 


P(x) = YIoDLe), 


where L,(x) is a polynomial of degree n given by 


A 
(15.9) L,(x) = Ax) 

A,(Xz) 
Thus, for each fixed x, P(x) is a linear combination of the prescribed values f(xo), f(x1), 
...,f(*,). The multipliers Z,(x) depend only on the points x), x,,..., x, and not on the 


prescribed values. They are called Lagrange interpolation coefficients. If we use the 
formulas in (15.6) and (15.7) we can write Equation (15.9) in the form 


(15.10) L,(x) = | [| ——~£. 
ra a 


This product formula provides an efficient method for evaluating the number L,(x) for 
a given x. 
Note: The Lagrange coefficients L;,(x) are often expressed in the form 
A,(x) 
L(x) == 5 


where A’ is the derivative of the polynomial in (15.5). To prove this formula it suffices to 
show that A’(x;,) = A;(x;,). Differentiating the relation 


A(x) = (x — xx)A;(x) 


we obtain A’(x) = (x — x,)A,(x) + A,(x). When x = x; this gives us A’(x;,) = A;(x,). 


EXAMPLE. Determine the polynomial of degree <3 that takes the values yo, yi, Yo, Vg 
at the points —2, —1, 1, 2, respectively. 


Solution. We take x» = —2, x, = —1, x, =1, x3 =2. The polynomials L,(x) in 
(15.10) are given by the formulas 


__ @+DG=De-2 ie 
Li) = yd a Nad ae t DE-DE =D, 


ee (x + 2)(x — 1)(x — 2) = 1 
(—1+2)(-1-—1)(-1-—2) 6 


_@4DEEDE-D_ 1 - 
“a Tee) | U6 
(x+2%+DG—-)_ 1 
(242(2+D)2—-1 12 


(x + 2)(x — I(x — 2), 


L;(x) = (x + 2)(x + 1)(x — 1). 
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Therefore the required polynomial is 


P(x) — + y,L,(X) + Yole(x) + y3l3(x) 


rie + 1)(x — 1)(x — 2) 44 5 (x + 2)(x — 1)(x — 2) 


de + (x + I(x — 2) 4 dees 2\(x + Ix — 1). 


To compute the value of P(x) for a specific x it is usually better to leave the polynomial 
in this form rather than to rewrite it in increasing powers of x. For example, if yp = —5, 
yi = 1, yo = 1, and ys = 7, the value of P(x) for x = 3 is given by 


P(2) = 2(2)G)(— 2) + 6G)(—-®) — «OG(—-4) + FOE) 


15.7 Equally spaced interpolation points 


In the foregoing discussion the interpolation points x), x,,..., X, were assumed to be 
distinct but otherwise arbitrary. Now we assume they are equally spaced and show that 
the bagrange coefficients L,(x) can be considerably simplified. Suppose x) < x, < 
X_<°*+ <x,, and let h denote the distance between adjacent points. Then we can write 


X; =X + jh 


forj =0,1,2,...,n. Since x, — x; = (kK —s)h, Equation (15.10) becomes 


(15.11) E(x) = [[2= PE 8 Ty eed, 
2 ee ay 

where 

X — Xo 


h 


i= 


In the last term on the right of (15.11) the product of the factors independent of ¢ is 


dP (py 1) pe 
al) Wea (dea) ae) Tp per: 
a Od 
k!(n — k)! n! (i) 


where ({) is the binomial coefficient. Since x = x) + th, Equation (15.11) now becomes 


_.1\"-k n 
(15.13) L,(Xo + th) at 7 () Ie —j). 


Error analysis in polynomial interpolation 583 


For each fixed n, the right member of (15.13) is a function of k and ¢ that can be tabulated. 
Extensive tables of the Lagrangian coefficients for equally spaced interpolation points have 
been prepared by the National Bureau of Standards. (See Reference 13 in the bibliography 
at the end of this chapter.) If x and A are chosen so that the number ¢ = (x — x )/h is one 
for which the Lagrangian coefficients L,(x 9 + th) are tabulated, the actual calculation of 
P(Xo + th) is reduced to a multiplication of the f(x,,) by the tabulated L(x) + th), followed 
by addition. 


15.8 Error analysis in polynomial interpolation 


Let f be a function defined on an interval [a, b] containing the n + 1 distinct points 
Xo, X1, +--+, X,, and let P be the interpolation polynomial of degree <n which agrees 
with f at these points. If we alter the values of f at points other than the interpolation 
points we do not alter the polynomial P. This shows that the function f and the poly- 
nomial P may differ considerably at points other than the interpolation points. If the 
given function f has certain qualities of “smoothness” throughout the interval [a, b] we 
can expect that the interpolating polynomial P will be a good approximation to f at points 
other than the x,. The next theorem gives a useful expression that enables us to study the 
error in polynomial interpolation when the given function has a derivative of order n + 1 
throughout [a, 5). 


THEOREM 15.3. Let x9, X1,...,X, ben + 1 distinct points in the domain of a function f, 
and let P be the interpolation polynomial of degree <n that agrees with f at these points. 
Choose a point x in the domain of f and let [«, B] be any closed interval containing the points 
Xo, %X1,+++,X,, and x. If f has a derivative of order n + | in the interval [«, B] there is at 
least one point c in the open interval («, B) such that 


_ _ A(x) (n+1) 
(15.14) f(x) EO ay. (c), 


where 
A(x) = (x — Xo)(x — X41) °° (xX — X,). 


Note: Point c depends on both x and n. 


Proof. If x is one of the interpolation points x,, then A(x,) = 0 and Equation (15.14) 
is trivially satisfied for any choice of c in (a, #). Suppose, then, that x is not one of the 
interpolation points. Keep x fixed and define a new function F on [«, 8] by the equation 


(15.15) F(t) = AMS — PO] — AML) — PO). 


The right-hand side of this equation, as a function of ¢, has a derivative of order n + 1; 
hence the same is true of the left-hand side. Since P(t) is a polynomial in ¢ of degree <n, 
its (n + 1)st derivative is identically zero. The polynomial A(t) has degree n + 1, the 
term of highest degree being t”*!, and we have A'"*)(t) = (n + 1)!. Therefore, if we 
differentiate Equation (15.15) n + 1 times with respect to t we obtain the formula 


(15.16) FOO (t) = AFM) — (2 + IIL) — PO). 
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From the definition in Equation (15.15) we see that F has the value zero at the n + 1 
interpolation points x9, x,,..., X, and also at the point x. Therefore F(t) = 0 at n + 2 
distinct points in the interval [«, 8]. These points determine n + 1 adjacent subintervals 
of [«, 6] and the function F vanishes at both endpoints of each of these subintervals. By 
Rolle’s theorem, the derivative F’(t) must be zero for at least one ¢ interior to each sub- 
interval. If we choose exactly one such ¢ from each subinterval we obtain n + 1 distinct 
points in the open interval (a, 6) at which F’(t) = 0. These points, in turn, determine n 
subintervals at whose endpoints we have F’(t)= 0. Applying Rolle’s theorem to F’ we 
find that the second derivative F’(t) is zero for at least n distinct points in («, 6). After 
applying Rolle’s theorem n + 1 times in this manner we finally find that there is at least 
one point c in (a, B) at which Ft) (c) =0. Substituting this value of c in Equation 
(15.16) we obtain 


(n + 1)! [f(@x) — P(x)] = AQ) ™(C), 


which is the same as (15.14). This completes the proof. 


It should be noted that, as with approximation by Taylor polynomials, the error term 
involves the (n + 1)st derivative f+ (c) evaluated at an unknown point c. If the extreme 
values of f'"t) in [«, 8] are known, useful upper and lower bounds for the error can be 
obtained. 

Suppose now that the interpolation points are equally spaced and that x) < x, < 
X2<+*'<x,. Ifh denotes the spacing we can write 


Xj = Xq + jh and X=Xo + th, 


where ¢ = (x — X9)/h. Since x — x; = (t — j)h, the polynomial A(x) can be written as 


A(x) = TT @ — x) =a TT —p). 
I= I= 
Formula (15.14) now becomes 


(15.17) f(x) — P(x) = a 7 pnt Ta 
with ¢ = (x — X))/h. 


EXAMPLE. Error in linear interpolation. Suppose a function f with a second derivative 
is tabulated and we wish to estimate its value at a point x intermediate to two consecutive 
entries xX) and x» +. If we use linear interpolation we approximate the graph of f over 
the interval [x9, Xx) + /] by a straight line, as shown in Figure 15.1. If P denotes the linear 
interpolating polynomial, the error estimate in (15.17) becomes 


(15.18) f(x) — P(x) -o h'x(t — 1), 


where t = (x — Xp)/h. When x lies between x, and x) + h we have 0 < t < 1 and the 
maximum value of |¢(¢ — 1)| in this interval is }. Therefore (15.18) gives us the estimate 


If) — P(x)| < LO" a a 
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Ficure 15.1 Linear interpolation. 


The point c is an unknown point in the interval (xp), x9 + /). If the second derivative 
f” is bounded in this interval, say | f”’(x)| < M, the error estimate becomes 


f(x) — P(X) < we | 


In particular, if fis a sine or cosine, then | f”(x)| < 1 for all x and we have | f(x) — P(x)| < 
h?/8. If a table of sines or cosines has entries for every degree (one degree = 7/180 
radians) we have h = 7/180, so 

h? 7 10 1 


= < < 
8 8(180)? 259,200 25,000 


= 0.00004. 


Since this error does not exceed § in the fourth decimal place, linear interpolation would be 
satisfactory in a four-place table. The error estimate can be improved in portions of the 
table where | f”(c)| is considerably less than 1. 


15.9 Exercises 


1. In each case find the polynomial P of lowest possible degree satisfying the given conditions. 
(a) P(—-1) =0, PO) =2, PQ) =7. 
(b) PU) =1, PQ) =0, PB) =0, P44 =1. 
(c) Pd) =1, PQ) =2, PGB) =3, PO) =1. 
(d) P(O) = —2, PU) =0, P(-1) = -—2, P(2) = 16. 
(e) P(—2) = 11, P(-1) = —-11, PO)=-S5, Pl) = -1. 

2. Let f(x) = cos (7x/4). Find the polynomial of smallest possible degree that takes the same 
values as fat the points —2, —#, 0, 3, 2. 

3. Let P be a polynomial of degree <n and let A(x) = (x — Xo)(x — x1): °* (x — Xp), where 
Xo. X15-++5X, aren + 1 distinct points. 
(a) Show that for any polynomial B the polynomial Q given by Q(x) = P(x) + A(x)B(x) 
agrees with P at the points x9, %1,..-,Xn- 
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(b) Prove also the converse. That is, if Q is any polynomial that agrees with P at the points 
Xo, Xy5+++5Xn, then Q(x) = P(x) + A(x)B(x) for some polynomial B. 
4. (a) Find the polynomial Q of lowest possible degree that satisfies the conditions 


Q.-2=-5, O-1I=-1, Q0)=1, QO=-1. 


[Hint: First find a polynomial P that takes the prescribed values at —2, —1, 1, 
and then use Exercise 3 to determine Q.] 


(b) Find the polynomial Q of lowest possible degree that satisfies the conditions in part (a) 
with Q’(0) = —3 instead of Q’(0) = —1. 

5. Let f(x) = log, x for x > 0. Compute P(32), where P is the polynomial of lowest possible 

degree that agrees with fat the points: 

(a) x = 1, 64. (c) x = 4, 16, 64. 

(b) x = 1, 16, 256. (d) x = 1, 4, 16, 64, 256. 

In each case compute the difference f(32) — P(32). These examples show that the accuracy in 
polynomial interpolation is not necessarily improved by increasing the number of interpolation 
points. 

6. The Lagrange interpolation coefficients L;,(x) given by Equation (15.10) depend not only on 
x but also on the interpolation points x9, x,,...,%X,. We can indicate this dependence by 
writing L,(x) = L,(x;X), where X denotes the vector in (7 + 1)-space given by X = 
(X9,X1,--+»,%Xn,). For a given real number 5, let b denote the vector in (n + 1)-space all of 
whose components are equal to 6. If a # 0, show that 


L,(ax +b; aX + 6) =L,(x;X). 


This is called the invariance property of the Lagrange interpolation coefficients. The next 
exercise shows how this property can be used to help simplify calculations in practice. 
7. Let P denote the polynomial of degree <4 that has the values 


P(2.4) = 72, P(2.5) = 30, P(2.7) = 18, P(2.8) = 24, P(3.0) = 180. 


(a) Introduce new interpolation points u,; related to the given points x by the equation 
u,; = 10x; — 24. The u; are integers. For each k = 0,1,2,3,4, determine the Lagrange 
interpolation coefficients L;,,(x) in terms of u, where u = 10x — 24. 

(b) Use the invariance property of Exercise 6 to compute P(2.6). 

8. A table of the function f(x) = log x contains entries for x = 1 to x = 10 at intervals of 0.001. 
Values intermediate to each pair of consecutive entries are to be computed by linear inter- 
polation. Assume the entries in the table are exact. 

(a) Show that the error in linear interpolation will not exceed § in the sixth decimal place. 
(b) For what values of x will linear interpolation be satisfactory for a seven-place table? 

(c) What should be the spacing of the entries in the interval 1 < x < 2 so that linear inter- 
polation will be satisfactory in a seven-place table? 


In Exercises 9 through 15, x9, x1,..., X, are distinct points and 


n n A 
Ax)=[J[@-x), A@=]]@--x), L(x) = at 
j=0 j=0 k\*k 
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9. Derive the formula A’(x) = pe A,(x) by use of (a) logarithmic differentiation; (b) 
Lagrange’s interpolation formula. 
10. Prove each of the following formulas: 


* Ax (x) 


; = 0 for all x. 
= A’ (xx) 


(a) > Leo) =1 and 
k=0 


“| 
(b) > —— = 0. [Hint: Use part (a) with suitable values of x.] 
Pe A (Xx) 


11. Let P be any polynomial of degree <n. Show that the coefficient of x” is equal to 
<P (Xz) 
pes A (Xx) 


12. (a) Determine a and b so that the polynomial 
P,(x) = {a + b(x — x,)} L;(x)? 
will have the following properties: 
P,(x;) = 0 for all i, Pi(x,) = 1, and Pi(x;) = 0 for ik. 
(b) Determine c and d so that the polynomial 
Ox(x) = {c + d(x — x4) } Ly (x)? 
will have the following properties: 
O,(x,) = 1, Q;,(x,;) = 0 for i#k, and OQ.(x,) =0 for alli. 


(c) Let A(x) = he F(X) Qe(x) + bars f' (x) P,(x), where f is a given function that is 
differentiable at x9, x,,...,X,- Prove that 


H(x;) =f(x;) and A(x) =f'(x,)  foralli. 


Prove also that there is at most one polynomial H(x) of degree <2n + 1 with this property. 
13. (a) Let P and Q be two polynomials of degree <n satisfying the n + 1 conditions 


P(xXo) = Q(%), P(x) = O'(™%), PP" (%) = O"(™.), we PP (Xn) = OM (,). 
Prove that P(x) = Q(x) for all x. 
(b) Let Bo(x) = 1, and for n > 1 define 


x(x —n)™1 
B, (x) = an” . 


Show that B/(x) = B,_,(x — 1) forn > 1 and deduce that 


B,(0) = BA) = Bh) =::+ =B"-Vin-1)=0 and BM (n) =1. 


588 Introduction to numerical analysis 


(c) Show that the one and only polynomial of degree <n satisfying the conditions 


PO=caq, PUW=q, P’R=a, ..., Pn =, 
is given by 
P(x) = > c,B,(x) ; 
=0 
(d) If x, =x) + khfork =0,1,2,...,”,whereh > 0, generalize the results in (b) and (c). 


14. Assume x9, X,,..., X, are integers satisfying x» < xy <-+-++ < Xp. 
(a) Prove that |A’(x;)| > k! (n — k)! and deduce that 


7 So 
tat [A (x;)| nN. 


(b) Let P be any polynomial of degree n, with the term of highest degree equal to x". Let M 
denote the largest of the numbers |P(%9)|, |P(x)|,... , |P(x,)|. Prove that M > n!/2". [Hint: 
Use part (a) and Exercise 11.] 


15. Prove the following formulas. In parts (a) and (b), x is any point different from x9, X,,...,Xn- 
A'(x) ~ 1 
(a) A(x) =>; — x; 
j=0 , 
mn A’(x) A(x) 1 Fi S 1 A(x) < 1 
A(x)  A’(x) ax — xX; Lax — X; A'(x) = (x — x,)? 
jth j#k jek 
A(x 1 
(c) “¢ ») _ > 
A (Xx) = Xp — X; 
j#k 
16. Let P,,(x) be the polynomial of degree <n that agrees with the function f(x) = eat then + | 
integers x = 0,1,...,m. Since this polynomial depends on a we denote it by P,(x; a). 
Prove that the limit 
, Pr(x; a) — 1 
lim ——--_-—— 
a—0 a 


exists and is a polynomial in x. Determine this polynomial explicitly. 


15.10 Newton’s interpolation formula 


Let P, denote the interpolation polynomial of degree <n that agrees with a given 
function fat n + | distinct points x), x,,..., x,. Lagrange’s interpolation formula tells 
us that 


P,(x) = TLCS ev | 


where L,(x) is a polynomial of degree n (the Lagrange interpolation coefficient) given by 
the product formula 


(15.19) L,(x) = ]7—=~, for k=0,1,2,...,n. 


J#k 
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Suppose we adjoin a new interpolation point x,,, to the given points x9, ¥,,...,%,- To 
determine the corresponding polynomial P,,, by Lagrange’s formula it is necessary to 
compute a new interpolation coefficient L,,, and to recompute all the earlier coefficients 
Ly, Ly, ..., L,, each of which is now a polynomial of degree n + 1. In practice this 
involves considerable labor. Therefore it is desirable to have another formula for deter- 
mining P, that provides an easier transition from P, to P,,,. One such formula was 
discovered by Newton; we shall derive it from the following theorem. 


THEOREM 15.4. Given n + 2 distinct points X9, X1,.---5Xn>%Xn41- Let P,, be the poly- 
nomial of degree <n that agrees with a given function f at X),...,X,, and let P,,,, be the 
polynomial of degree <n + 1 that agrees with f at Xo, X14, ~--5 Xn» Xn4i- Then there is a 
constant C,,,, uniquely determined by f and by the interpolation points Xo, ... 5 Xn41, Such 
that 
(15.20) Pryi(X) = Pa(X) + Cutie — Xo) + (X — X,). 


Proof. Let O(x) = P,(x) + c(x — X9)-°* (x — x,), where c is an unspecified constant. 
Then Q is a polynomial of degree <n + 1 that agrees with P,, and hence with fat each of 
then + | points x9,...,x,. Now we choose c to make Q agree with falso at x,,,. This 
requires 

ST (Xn41) a P,(Xn41) or C(Xn41 _ Xo) as (Xn41 _ an 


Since the coefficient of c is nonzero, this equation has a unique solution which we call 
Cn4i- Taking c =c,,,, we see that Q = P,,,,. 


The next theorem expresses P,,(x) in terms of the numbers c,, ..., c,. 
THEOREM 15.5. NEWTON’S INTERPOLATION FORMULA. If X9,...,X, are distinct, we have 
n 
(15.21) P,(x) = f (Xo) + > x(x =X q) O(N ea) 
k=1 


Proof. We define Po(x) = f(x») and take nm = 0 in (15.20) to obtain 


P(x) = f (Xo) + C1(x — Xo). 
Now take n = | in (15.20) to get 
P(x) = Py(x) + Co(x — Xo)(x — x1) = f(%q) + Cr(X — Xo) + C2(% — Xo)(X — X4). 
By induction, we obtain (15.21). 


The property of Newton’s formula expressed in Equation (15.20) enables us to calculate 
P,»+1 simply by adding one new term to P,,. This property is not possessed by Lagrange’s 
formula. 

The usefulness of Newton’s formula depends, of course, on the ease with which the 
coefficients c,, C,,..., ¢, can be computed. The next theorem shows that c,, is a linear 
combination of the function values f(x), ..., f(x,). 
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THEOREM 15.6. The coefficients in Newton’s interpolation formula are given by 


(15.22) C. oe. where A,(Xx;,) = ITO — x,). 


Proof. By Lagrange’s formula we have 


P(x) =¥ Ly) f), 


where L,(x) is the polynomial of degree n given by (15.19). Since the coefficient of x” in 
L,(x) is 1/A,(x;,), the coefficient of x” in P,,(x) is the sum appearing in (15.22). On the other 
hand, Newton’s formula shows that the coefficient of x" in P,(x) is equal to c,. This 
completes the proof. 


Equation (15.22) provides a straightforward way for calculating the coefficients in 
Newton’s formula. The numbers A,(x,) also occur as factors in the denominator of the 
Lagrange interpolation coefficient L,(x). The next section describes an alternate method 
for computing the coefficients when the interpolation points are equally spaced. 


15.11 Equally spaced interpolation points. The forward difference operator 


In the case of equally spaced interpolation points with x, = x5 + kh fork =0,1,...,n 
we can use Equation (15.12) to obtain 


1 ™ 1 ie te (=) */n 
i 7 vers ! (i) 


J#k j 


In this case the formula for c, in Theorem 15.6 becomes 


Te haul (;) £00). 


The sum on the right can be calculated in another way in terms of a linear operator A 
called the forward difference operator. 


(15.23) Cc, = 


DEFINITION. Let h be a fixed real number and let f be a given function. The function 
Af defined by the equation 


Af(x) = f(x + A) — f(x) 


is called the first forward difference of f. It is defined at those points x for which both x and 
x + hare in the domain of f. Higher order differences Af, A*f, . . . are defined inductively as 
follows: 

Artf= A(A*f) for k =1,2,3,.... 


Note: Thenotations A, f(x) and Af(x; h) are also used for Af(x) when it is desirable to 
indicate the dependence on h. It is convenient to define A°f = f. 
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The nth difference A"f(x) is a linear combination of the function values f(x), f(x + A), 
., f(x + nh). For example, we have 


A*f(x) = {f(x + 2h) — f(x + AD} — Tf + 4) — fO} 


= f(x + 2h) — 2f(x + h) + f(x). 
In general, we have 


(15.24) Af) => (-)"4 (;) f(x + kh). 
k 
k=0 
This is easily proved by induction on n, using the law of Pascal’s triangle for binomial 


coefficients: 
n n\ (n+l 
(.71) + () = ("E'): 


Now suppose fis defined at n + 1 equally spaced points x, = x) + khfork =0,1,..., 
n. Then from (15.23) and (15.24) we obtain the formula 


1 
C.. = Af (Xo). 
a F(X) 


This provides a rapid method for calculating the coefficients in Newton’s interpolation 
formula. The diagram in Table 15.1, called a difference table, shows how the successive 
differences can be systematically calculated from a tabulation of the values of fat equally 
spaced points. In the table we have written f, for f(x;,). 

Newton’s interpolation formula (15.21) now becomes 


= A* A'f (Xo) (xo) 4 
(15.25) P,(x) = f(%0) ‘Zoo nae Ho - 
TABLE 15.1 
x | f(x) Af(x) A?f(x) AP f(x) 
Xo | fo 
\ 
fi —fo = Af) 
7 Ny 
x | fj Af (xy) — Af (xo) = A®f (x9) 
N ZA “Ny 
fr =f Af (x) A*f(x,) a A*f (Xp) = A*f (Xp) 
7 Ny 7 
xX. | fe Af (Xs) — Af (x,) = A? f(x) 
ras 


N 
fs — fo = Af(%s) 
7 

x3 | fs 
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If we write 


k—-1 k—1 : BA iy xy es . 
Tx) = [1 — xy — sn) = TT (F583) = Ta, 
j=0 j=0 j=0 h j=0 
where t = (x — Xo)/h, Equation (15.25) becomes 
S A“f (x0) TT 
(15.26) P,(x) = f(%0) +> =" Ta — A). 
ot k! j=0 
15.12 Factorial polynomials 
The product t(t — 1)+ ++ (¢ — k + 1) which appears in the sum in (15.26) is a polynomial 


in ¢ of degree k called a factorial polynomial, or the factorial kth power of t. It is denoted 
by the symbol ¢™. Thus, by definition, 


k—-1 


i || (9), 


j=0 


We also define ¢ = 1. If we consider the forward difference operator A with h = 1, 
that is, Af(x) = f(x + 1) — f(x), we find that 


At = nt) for n> 1. 


This is analogous to the differentiation formula Dt” = nt"—* for ordinary powers. Thus, 
the factorial power ¢‘”) is related to differences much in the same way that the ordinary 
power ¢” is related to derivatives. 

With the use of factorial polynomials, Newton’s interpolation formula (15.26) becomes 


n k 
P,(x» + th) => = 1), 
k=0 


Expressed in this form, Newton’s formula resembles the Taylor formula for the poly- 
nomial of degree <n that agrees with f and its first m derivatives at x). If we write 


t)} @@  tt—1)---@-—k4+ 1 
(J-ae k! 


Newton’s formula takes the form 
St 
Pa(%o + th) => (;,) AC). 
k=0 


Further properties of factorial polynomials are developed in the following exercises. 
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15.13 Exercises 
1. Let Af(x) = f(x +h) — f(x). If fis a polynomial of degree n, say 


fo) = Sax" 


r=0 


with a, #0, show that (a) A*f(x) is a polynomial of degree n —k ifk <n; (b) A*f(x) = 
ni h"a,; (c) A*f(x) = Ofork >n. 
2. Let Af(x) = f(x +h) — f(x). If f(x) = sin (ax + 5), prove that 


— ah\® . nah + na 
A*f(x) = 2 sin > sin | ax a aa : 


3. Let Af(x) = f(x +h) — f(x). 
(a) If f(x) = a, where a > 0, show that A*f(x) = (a” — 1)*a®. 
(b) If g(x) = (1 + a)*, where a > 0, show that A*g(x) = a*g(x). 
(c) Show that the polynomial P, of degree n that takes the values P,(k) = (1 + a)* for k = 
0,1,2,...,m is given by 


n ak 
P,(x) = > Fx, 
k=0 


4. Let x'”) be the factorial nth power of x. Since x is a polynomial in x of degree n with the value 
0 when x = 0, wecan write 


n 
x SS Seiax”. 
k=1 


The numbers S;, ,, are called Stirling numbers of the first kind. From the definition of x) it is 
clear that S, , =1forn >0. 

(a) Show that S,_ 1, = —n(n — 1)/2 and that S,, = (—1)"4@ — 1)! forn 21. 

(b) Prove that Sy n41 = Sp_3.n — 2Sy,. Use this relation to verify the entries in Table 15.2, 
a table of Stirling numbers of the first kind, and construct the next three rows of the table. 


TABLE 15,2 
n Sin Soin S3in Sain S5.n Sein Son 
1 1 
2 —1 1 
3 2 —3 1 
4 —6 11 —6 1 
5 24 —50 35 —10 1 
6 —120 274 —225 85 —15 1 
7 


720 —1764 1624 —735 175 ~21 1 


(c) Express the polynomial x + 3x") + 2x] + 1 as a linear combination of powers of x. 
5. (a) Prove that 


ea, exon), x ap 3x6) 48), 
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and that, in general, 
n k 
ne > BLO) 
k! 


k=1 


bd 


where f(x) =x" and Af(x) = f(x + 1) — f(x). The numbers 7, = A¥f(0)/k! are called 
Stirling numbers of the second kind. 
(b) Prove that 


Ayn! = (x +k) A®x” + k A®-1x” 


and use this to deduce that Ty ny = Tyy,.n + Tu n- 
(c) Use the recursion formula in part (b) to verify the entries in Table 15.3, a table of Stirling 
numbers of the second kind, and construct the next three rows of the table. 


TABLE 15.3 


1 1 

2 1 1 

3 l 3 1 

4 1 7 6 1 

5 1 15 25 10 1 

6 1 31 90 65 15 

7 1 63 301 350 140 21 1 


(d) Express the polynomial x* + 3x? + 2x — 1 as a linear combination of factorial poly- 
nomials. 
6. (a) If p is a positive integer and if a and 5 are integers with a < 5, prove that 


= . ite — q(Ptt) 
k\P : 
Po pt pei 


This formula is analogous to the integration formula for J° x? dx. It should be noted, however, 
that the upper limit in the sum is b — 1, not b. 
(b) Verify that k(k + 3) = 4k) +k). The use part (a) to show that 


n (2) 1)(3) 1 
Sek +3 gS tO SEO -weer 
k=1 


(c) If f(k) is a polynomial in k of degree r, prove that 


> £&) 


k=1 


is a polynomial in n of degree r + 1. 
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7. Use the method suggested in Exercise 6 to express each of the following sums as a polynomial 


in 7. 

(a) ¥ (42 +7k +6). (c) Sk(k + IK +2). 
k=1 k=1 

(b) 5 K(k +1). (d) ¥ A. 
k=1 k=1 


8. Let A denote the linear operator defined by the equation 
A(f) = ay Af +a, Af +--+ + Gn Af + anf, 


where a), @,,...,4, are constants. This is called a constant-coefficient difference operator. 
It is analogous to the constant-coefficient derivative operator described in Section 6.7. With each 
such A we can associate the characteristic polynomial p, defined by 


P4lr) = aor” + ayr™) + -°+ tana tay. 


Conversely, with every polynomial p we can associate an operator A having this polynomial as its 
characteristic polynomial. If A and B are constant-coefficient difference operators and if A is a 
real number, define A + B, AB, and 4A by the same formulas used in Section 6.7 for derivative 
operators. Then prove that Theorem 6.6 is valid for constant-coefficient difference operators. 


15.14 A minimum problem relative to the max norm 


We consider a problem that arises naturally from the theory of polynomial interpolation. 
In Theorem 15.3 we derived the error formula 


_ — A(X) pina) 
(15.27) f(x) — P(x) = oa ni (c), 
where 
A(x) = (x — Xp)(X — ¥y) °° (X — x,). 


Here P is the unique polynomial of degree < n that agrees with fat n + 1 distinct points 
Xo, X1,---, X, in [a, b]. The function fis assumed to have a derivative of order n + 1 
on [a, b], and c is an unknown point lying somewhere in [a, 6]. To estimate the error in 
(15.27) we need bounds for the (n + 1)st derivative f'"*) and for the product A(x). Since 
A is a polynomial, its absolute value has a maximum somewhere in the interval [a, 5]. 
This maximum will depend on the choice of the points x9, x,,..., X,, and it is natural to 
try to choose these points so the maximum will be as small as possible. 
We can denote this maximum by ||A||, where || A|| is the max norm, given by 


|Al| = max |4(x)]. 


a< 


The problem is to find a polynomial of specified degree that minimizes || A||. This problem 
was first solved by Chebyshev; its solution leads to an interesting class of polynomials that 
also occur in other connections. First we give a brief account of these polynomials and then 
return to the minimum problem in question. 
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15.15 Chebyshev polynomials 


Let x + iy be a complex number of absolute value 1. By the binomial theorem we have 


n 


(+ => ({) x? 


k=0 


for every integer n > 0. In this formula we write x = cos 0, y = sin 6, and consider the 
real part of each member. Since 


(x + iy)” = (cos 6 + isin 0)” = e’”? = cos nO + isin nd, 


the real part of the left member is cos n6. The real part of the right member is the sum over 
even values of k. Hence we have 


(15.28) cos nO = x” — (5) xr ty? (1) xrayt# — free, 


Since y? = sin? 6 = | — cos? 6 = | — x?, the right-hand member of (15.28) is a poly- 
nomial in x of degree n. This polynomial is called the Chebyshev polynomial of the first 
kind and is denoted by 7,,(x). 


DEFINITION. The Chebyshev polynomial T,,(x) is defined for all real x by the equation 


{[n/2] 


T,(x) = > (>) xrPh (2 yh 


k=0 


From Equation (15.28) we obtain the following theorem. 


THEOREM 15.7. If —1 <x <1 we have 
T,,(x) = cos (m arccos x). 
Proof. If 0 = arccos x then x = cos 6 and T,,(x) = cos nO. 


The Chebyshev polynomials can be readily computed by taking the real part of (x + iy)” 
with y? = 1 — x?, or by using the following recursion formula. 


THEOREM 15.8. The Chebyshev polynomials satisfy the recursion formula 
T41(X) = 2xT,,(x) _ T,,-1(x) for n = ls 


with T,(x) = 1 and T,(x) = x. 
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Proof. First assume —1 < x < 1 and put x = cos 0 in the trigonometric identity 
cos (n + 1)6 + cos (n — 1)6 = 2 cos 6 cos nO. 


This proves that 7,,,,(x) + T,_1(x) = 2xT,,(x) for x in the interval -1 <x <1. But 
since both members are polynomials, this relation must hold for all x. 


The next five polynomials are 


T(x) = 2x? — 1, T3(x) = 4x° — 3x, T,(x) = 8x? — 8x? + 1, 
T;(x) = 16x5 — 20x3 + 5x, T,(x) = 32x® — 48x4 + 18x? — 1. 


The recursion formula shows that all the coefficients of 7,,(x) are integers; moreover, 
the coefficient of x” is 271. 

The next theorem shows that 7,,(x) has exactly n first order zeros and that they all lie in 
the interval [—1, 1]. 


THEOREM 15.9. Ifn > 1 the polynomial T,,(x) has zeros at the n points 


iy = cos EU, ee 0: 1 26231. 
n 


Hence T,,(x) has the factorization 
n—1 
T, (x) = 2”(x — x9)(x — x1) °° (X — X__y) = 2771 YT (x — cos —— 
k=0 n 


Proof. We use the formula T,(x) = cosn@. Since cos nO = 0 only if n@ is an odd 
multiple of 7/2, we have T,,(x) = 0 for x in [—1, 1] only if m arccos x = (2k + 1)z/2 for 
some integer k. Therefore the zeros of T,, in the interval [—1, 1] are to be found among the 
numbers 


(15.29) Sates. PS a. 
n 2 
The values k = 0,1, 2,...,2— 1 given distinct zeros xy, X,,..., X,_1, all lying in the 


open interval (—1,1). Since a polynomial of degree n cannot have more than n zeros, 
these must be al/ the zeros of 7,,. The remaining x, in (15.29) are repetitions of these n. 


THEOREM 15.10. Jn the interval [—1, 1] the extreme values of T,,(x) are +1 and —1, 
taken alternately at the n + 1 points 


(15.30) en a a 
n 


Proof. By Rolle’s theorem, the relative maxima and minima of 7, must occur between 
successive zeros; there are n — | such points in the open interval (—1, 1). From the cosine 
formula for T,, we see that the extreme values, +1, are taken at the nm — 1 interior points 
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T; (x) 


FiGurE 15.2 Graphs of Chebyshev polynomials over the interval [—1, 1]. 


cos (km/n), kK = 1, 2,..., n — 1, and also at the two endpoints x = 1 and x = —1. 
Therefore in the closed interval [—1, 1] the extreme values +1 and —1 are taken alter- 
nately at the n + 1 points fo, 4,..., ¢, given by t, = cos (kz/n) fork =0,1,2,...,7n. 


Figure 15.2 shows the graphs of 7,,..., 7; over the interval [—1, 1]. 


15.16 A minimal property of Chebyshev polynomials 


We return now to the problem of finding a polynomial of a specified degree for which the 
max norm is as small as possible. The problem is solved by the following theorem. 


THEOREM 15.11. Let p,(x) = x" +.-°: be any polynomial of degree n > 1 with leading 
coefficient 1, and let 


IP»l| = max |p,(x)|. 
—-1s¢v<1 
Then we have the inequality 
(15.31) Pall = Wall, 


where T,,(x) = T,(x)/2"-1. Moreover, equality holds in (15.31) if P, = T,. 


Proof. In the interval [—1, 1] the polynomial 7, takes its extreme values, 1/2”-! and 
—1/2"-1, alternately at the n + 1 distinct points ¢, in Equation (15.30). Therefore ||7,,|| = 
Lat. 

We show next that the inequality 

1 


(15.32) [Pall <5 


leads to a contradiction. Assume, then, that p,, satisfies (15.32) and consider the difference 
r(x) = T(x) = P(x) : 
At the points ¢, given by (15.30) we have 


1 
qn-l 


_4\k 
PO ee 


ant ee ay oe k 
<r — Pulte) = ( | 


= (Dstt | , 
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Because of (15.32) the factor in square brackets is positive. Therefore r(¢,) has alternating 
signs at the n + 1 points tf), t,,...,¢,. Since ris continuous it must vanish at least once 
between consecutive sign changes. Therefore r has at least n distinct zeros. But since ris a 
polynomial of degree <n — 1, this means that r is identically zero. Therefore P, = T.,,, 
so ||P,,|| = |7,|| = 1/2", contradicting (15.32). This proves that we must have ||p,.|| > 
1/2"-1 = ||T,,|). 


Although Theorem 15.11 refers to the interval [—1, 1] and to a polynomial with leading 


coefficient 1, it can be used to deduce a corresponding result for an arbitrary interval [a, 5] 
and an arbitrary polynomial. 


THEOREM 15.12. Letq,(x) = c,x" + °°: be any polynomial of degree n > 1, and let 


lanl| = max |q,(x)|. 
a<x<b 
Then we have the inequality 
(b — a)" 


q2n-1 


(15.33) lanll = len 


Moreover, equality holds in (15.33) if 
(b — a)" eee 
(=, As 
Proof. Consider the transformation 


_2x—-—a—b 
b—a 


t 


This maps the interval a < x < 5 in a one-to-one fashion onto the interval -l <t<1. 
Since 


b—a b+a 
x = ——t+—— 
2 2 
we have 
n b—a\, 
= (+) t” + terms of lower degree, 
hence 


Gn(X) = Cy (*) p.0. 


where p,,(t) is a polynomial in t of degree n with leading coefficient 1. Applying Theorem 
15.11 to p, we obtain Theorem 15.12. 


15.17 Application to the error formula for interpolation 


We return now to the error formula (15.27) for polynomial interpolation. If we choose 
the interpolation points x9, x,,...,X, to be then + 1 zeros of the Chebyshev polynomial 
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T,,-, we can write (15.27) in the form 


f(x) — P(x) = en) gorr(¢9, 


2"(n + 1)! 
The points x9, x,,..., X, all lie in the open interval (—1, 1) and are given by 
vy = cos (5) for k=0,1,2,...,n. 
n+1 2 


If x is in the interval [—1, 1] we have |T,,,,(x)| <1 and the error is estimated by the 
inequality 


— ae eer (n+1) 
If(x) — P(x) < n+)! pe"). 


If the interpolation takes place in an interval [a, 5] with the points 


b—a b+t+a 
Xp + 
a 2 


as interpolation points, the product 


A(x) = (X — yo)(X — yi) ** + (X — Yn) 


satisfies the inequality |A(x)| < (b — a)"**/2?"*" for all x in [a,b]. The corresponding 
estimate for f(x) — P(x) Is 


(b pa ay 


SO) — POS sea 


| fF (c)| 


15.18 Exercises 
In this set of exercises 7,, denotes the Chebyshev polynomial of degree n. 


1. Prove that 7,(—x) = (—1)"T,,(x). This shows that 7, is an even function when n is even 
and an odd function when n is odd. 


2. (a) Prove that in the open interval (—1, 1) the derivative T, is given by the formula 


nsin nO 


T(x) = : where 6 = arccos x. 


sin 0 
(b) Compute 7, (1) and 7/(—1). 
Trii(x) — Tn41(0) T(x) — T,,-1(0) 


* 1 
3. Prove that | T,,(u) du = D) [a = ot a if n>2. 


4, (a) Prove that 27,,(x) T(x) = TingnX) + Timn_n(x). 

(b) Prove that Ting(x) = TmlT,(~)] = T,[Tm(x)]. 
5. If x = cos 6, prove that sin 6 sin 78 is a polynomial in x, and determine its degree. 
6. The Chebyshev polynomial 7, satisfies the differential equation 


(1 — x?)y” — xy’ + n’y =0 


10. 


11. 


12. 


Exercises 601 


over the entire real axis. Prove this by each of the following methods: 
(a) Differentiate the relation T(x) sin 6 = n sin n@ obtained in Exercise 2(a). 
(b) Introduce the change of variable x = cos 9 in the differential equation 


d? (cos n@) _ 


aD —n* cos né. 


. Determine, in terms of Chebyshev polynomials, a polynomial Q(x) of degree <n which best 


approximates x”*! on the interval [—1, 1] relative to the max norm. 


. Find a polynomial of degree <4 that best approximates the function f(x) = x° in the interval 


[0, 1], relative to the max norm. 


. A polynomial P is called primary if the coefficient of the term of highest degree is 1. For a 


given interval [a, b] let ||P|| denote the maximum of |P| on [a, 5]. Prove each of the following 

statements: 

(a) If b —a <4, for every « > 0 there exists a primary polynomial P with ||P|| < «. 

(b) If for every « > 0 there exists a primary polynomial P with ||P|| < «, thenb—a <4. 
In other words, primary polynomials with arbitrarily small norm exist if, and only if, the 

interval [a, b] has length less than 4. 

The Chebyshev polynomials satisfy the following orthogonality relations: 


0 if n#m, 


es o8 ee i wT if n=m=0, 


= V1 — x? 


if n=m>O. 


NI a 


Prove this by each of the following methods: 
(a) From the differential equation in Exercise 6 deduce that 


T(x) ce 


Tmn(*) Fe 5 (Vi # — x? 71) +7? “gaa 


Write a corresponding formula with n and m interchanged, subtract the two equations, and 
integrate from —1 to 1. 
(b) Use the orthogonality relations 


0 if n#m, n>0, m>OdO, 


if n=m=0, 


3 


TT 
| cos m8 cos né dé = 
0 


if n=m>0O, 


and introduce the change of variable x = cos 6. 
Prove that for —1 <x < 1 we have 


Saree (J . x2)n-V2 
xX 


Let yi, Yo,-++ +s Yn ben real numbers, and let 


2k — 1 
Pe cca for K = 1,22¢5.40 
2n 
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Let P be the polynomial of degree <n — 1 that takes the value y, atx, for] <k <n. Ifxis 
not one of the x, show that 


P(x) = - 2 (-1)F4y, Jt — x28 


T,,(x) 
ky — x}, 


13. Let P be a polynomial of degree <n — 1 such that 


J1 — x? |P(x)| <1 


for —1 <x <1. Provethat||P|| < 7, where || P|| is the maximum of |P| on the interval [—1, 1]. 


[Hint: Use Exercise 12. Consider three cases: x, <x <1; —-1 <x < xy; 
Xn <X <x ,; in the first two use Exercise 15(a) of Section 15.9. In the third 


case note that a 1 — x? > sin (7/2n) > 1/n.] 
In Exercises 14 through 18, U,(x) = T,/,(x)/(2 + 1) for n =0, 1, 2,. 


14. (a) Prove that U,,(x) = 2xU,_,(x) — Un_.(x) forn > 2. 
(b) Determine the explicit form of the polynomials Up), U,,..., Us. 
(c) Prove that |U,(x)| <n +1if-1<x <1. 

15. Show that U,, satisfies the differential equation 


(1 — x*)y” — 3xy’ +n(n + 2)y =0. 


16. Derive the orthogonality relations 


, 0 if mn 
| J1 — x2 Un (x)U,(x) dx = { , 
—1 ~ if m=n 
17. Prove that 
——— 2"(n + 1)! d” 
— 2 ae a ees 1 — x2)"+1/2 , 
Vi EUG) (=D Oa ae oe 
18. Let yy, Yo...» 5 Yn be m real numbers and let 
X;, = COS for k = 1,2,...,”. 
n+ 1 


Let P be the polynomial of degree <n — 1 that takes the value y, at x,forl <k <n. Ifx is 
not one of the x; show that 


PQ) = 73P XG DG = 2Dy, 


15.19 Approximate integration. The trapezoidal rule 


Many problems in both pure and applied mathematics lead to new functions whose 
properties have not been studied or whose values have not been tabulated. To satisfy 
certain practical needs of applied science it often becomes necessary to obtain quantitative 


Approximate integration. The trapezoidal rule 603 


a ath a + 2h a a+nh = b 


FiGuRE 15.3. The trapezoidal rule obtained by piecewise linear interpolation. 


information about such functions, either in graphical or numerical form. Many of these 
functions occur as integrals of the type 


F(x) = |" fat, 


where the integrand fis given by an explicit analytic formula or is known in part by tabular 
data. The remainder of this chapter describes some of the most elementary methods for 
finding numerical approximations to such integrals. The basic idea is very simple. We 
approximate the integrand f by another function P whose integral is easily computed, and 
then we use the integral of P as an approximation to the integral of f- 

If fis nonnegative the integral {° f(x) dx represents the area of the ordinate set of f over 
[a, b]. This geometric interpretation of the integral immediately suggests certain proce- 
dures for approximate integration. Figure 15.3 shows an example of a function f with 
known values at n + 1 equally spaced points a,a+h,a+2h,...,a+nh=b, where 
h=(b—a)[n. Letx,=a+kh. For eachk =0,1,2,...,m— 1 the graph of f over 
the interval [x;,, x,,,] has been approximated by a linear function that agrees with f at the 
endpoints x, and x,,,;. Let P denote the corresponding piecewise linear interpolating 
function defined over the full interval [a, b]. Then we have 


X — Xz 


(15.34) P(x) = “BL f(xy) + f(r) if Xp SX < Xp. 


Integrating over the interval [x,, X;,41] we find that 


{ P(x) dx tei pO EI ; 


When / is positive this is the area of the trapezoid determined by the graph of P over 
[X45 X41]. The formula holds, of course, even if fis not positive everywhere. Adding the 
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integrals over all subintervals [x,, x,.:] we obtain 


(15.35) | P(x) dx = > Ou) + fOr) 


n—1 


_ A Hayé 2D. f(a + kh) +4) | 


To use this sum as an approximation to the integral [? f(x) dx we need an estimate for 
the error, |° f(x) dx — J° P(x) dx. If f has a continuous second derivative on [a, 5] this 
error is given by the following theorem. 


THEOREM 15.13. TRAPEZOIDAL RULE. Assume f has a continuous second derivative f" 
on [a, b]. If nis a positive integer, let h = (b — a)/n. Then we have 


R _b-a ~ _ (b— a)’ ,, 
1536) | f(s)dx =" = *(s@) + 2D fla + kh) + f(b)) P= pro 


for some c in [a, b}. 


Note: Equation (15.36) is known as the trapezoidal rule. Theterm —f”(c)(b — a)?/12n? 
represents the error in approximating f° f(x) dx by J° P(x) dx. Once the maximum value 
of f” on [a, 5] is known we can approximate the integral of f to any desired degree of 
accuracy by taking n sufficiently large. Note that no knowledge of the interpolating 
function P is required to use this formula. It is only necessary to know the values of fat the 
points a,a +h,...,a@+nh, and to have an estimate for | f"(c)|. 


Proof. Let P be the interpolating function given by (15.34), where x, = a + kh. Ineach 
subinterval [x,, x,41] we apply the error estimate for linear interpolation given by Theorem 
15.3 and we find 


(15.37) f(x) — Plx) = (x — u(x — xe) ae 


for some ¢, IN (X;, Xz41). Let M, and m, denote the maximum and minimun, respectively, 
of f” on [a, b], and let 


B(x) = (% — X,)(%pr1 — X)/2. 
Then B(x) > 0 in the interval [x,, X,4:], and from (15.37) we obtain the inequalities 


m,B(x) < P(x) — f(x) < M2B(x) 


in this interval. Integrating, we have 


Leth xreth Leth 
(15.38) " I B(x) dx < i [P(x) — f(x)] dx < M, i” B(x) dx. 


The integral of B is given by 


3 


[“"s ; 1 [eth 1 4 h 
x)dx =- _ —x)dx=- t((h—t)dt=—. 
[Beas = 5 [. (¢ = AO — X)dx = 3 { (h— 1dr == 
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Therefore the inequalities (15.38) give us 


m, < - | oP) — fC) dx < My. 


vk 


Adding these inequalities for k = 0, 1, 2,...,m — 1 and dividing by n, we obtain 


ms <= [ (PG) — foo] de < My 
nh a 


Since the function f” is continuous on [a, 5], it assumes every value between its minimum 
m, and its maximum M, somewhere in [a, b]. In particular, we have 


, 12 (° 
fo) =< | (PG) - Fo] dx 
nh? Ja 
for some c in [a, 6]. In other words, 


| Foee [ POdee = f"(c). 


Using (15.35) and the relation A = (6 — a)/n we obtain (15.36). 


To derive the trapezoidal rule we used a linear polynomial to interpolate between each 
adjacent pair of values of f/ More accurate formulas can be obtained by interpolating with 
polynomials of higher degree. In the next section we consider an important special case 
that is remarkable for its simplicity and accuracy. 


15.20 Simpson’s rule 


The solid curve in Figure 15.4 is the graph of a function f over an interval [a, b]. The 
mid-point of the interval, (a + b)/2, is denoted by m. The dotted curve is the graph of a 
quadratic polynomial P that agrees with f at the three points a, m, and b. If we use the 
integral {° P(x) dx as an approximation to f° f(x) dx we are led to an approximate inte- 
gration formula known as Simpson’s rule. 

Instead of determining P explicitly, we introduce a linear transformation that carries the 
interval [a, b] onto the interval [0,2]. If we write 

x—a 


SS or x=a+(m-—a)t, 
m—a 


we see that ¢ takes the values 0, 1, 2 when x takes the values a, m, b. Now let 
p(t) = Pla + (m — a)t]. 


Then p(t) is a quadratic polynomial in ¢ that takes the values P(a), P(m), P(d) at the points 
t = 0, 1, 2, respectively. Also, we have 


mM—-QadJa 
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a m b 


FiGureE 15.4 Interpolation by a quadratic polynomial P. 


hence 
b 2 b—a 2 
(15.39) { P(x) dx =(m — a) | g(t) dt = ; { y(t) dt. 
a 0 0 
Now we use Newton’s interpolation formula to construct g. We have 
A?’ (0) 
Ht) = 9(0) + 1A9(0) + (r=, 


where Ag(t) = y(t + 1) — o(t). Integrating from 0 to 2 we obtain 
in y(t) dt = 2¢(0) + 2Ay(0) + 4 A*G(0). 
Since Ag(0) = ¢(1) — ¢(0) and A2¢(0) = ¢(2) — 2¢(1) + (0), the integral is equal to 
[F e@ dt = $e) + 491) + 9(2)] = MPCa) + 4P(m) + P(O)]. 


Using (15.39) and the fact that P agrees with f at a, m, b, we obtain 


(15.40) { P(x) dx =? —* [f(a + 4f(rn) + FB], 
Therefore, we may write 
| F(x) dx = P= [f(a) + 4f(mm) + $(B)] + R, 


where R = f? f(x) dx — f® P(x) dx. 
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If fis a quadratic polynomial, P is identical to f and the error R is zero. It is a remark- 
able fact that we also have R = 0 when fis a cubic polynomial. To prove this property we 
use the error estimate for Lagrange interpolation given by Theorem 15.3, and we write 


(15.41) f(x) — P(x) = (x — a)(x — m)\(x — yi ; 
where c € (a, b). When fis a cubic polynomial the third derivative f” is constant, say 
f" (x) = C, and the foregoing formula becomes 


f(x) — P(x) = E(x — a(x — myx = b) = E(t + hye — hi), 
where t = x — mand h = (b — a)/2. Therefore 
b _C h 3 22 _ 
R= | Lf) — Poy)dx =E [ ¢ h*t)dt=0, 


since the last integrand is an odd function. This property is illustrated in Figure 15.5. The 
dotted curve is the graph of a cubic polynomial f that agrees with P at a, m, b. In this 
case R = f° [f(x) — P(x)] dx = A, — Az, where A, and A, are the areas of the two shaded 
regions. Since R = 0 the two regions have equal areas. 

We have just seen that Equation (15.40) is valid if P is a polynomial of degree <3 that 
agrees with f at a, m, and b. By choosing this polynomial carefully we can considerably 
improve the error estimate in (15.41). We have already imposed three conditions on P, 
namely, P(a) = f(a), P(m) = f(m), P(b) = f(b). Now we impose a fourth condition, 
P'(m) = f'(m). This will give P and f the same slope at (m, f(m)), and we can hope that 
this will improve the approximation of f by P throughout [a, 5]. 


a m b 


FiGurE 15.5 The two shaded regions have equal areas for every cubic interpolating 
polynomial. 
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To show that such a P can always be chosen, we let Q be the quadratic polynomial that 
agrees with fat a, m, b, and we let 


P(x) = Q(x) + A(x — a)(x — m)(x — Bb), 


where A is a constant to be determined. For any choice of A, this cubic polynomial P 
agrees with Q and hence with f at a, m, b. Now we choose A to make P’(m) = f'(m). 
Differentiating the formula for P(x) and putting x = m we obtain 


P'(m) = Q'(m) + A(m — a)(m — b). 
Therefore if we take A = [f'(m) — Q’(m)]/[(m — a)(m — b)] we also satisfy the condition 
P'(m) = f'(m). 


Next we show that for this choice of P we have 


(15.42) f(x) — P(x) = (x — a)(x — m)*(x — pO 


for some z in (a, b), provided that the fourth derivative f exists in [a, b]. To prove 
(15.42) we argue as in the proof of Theorem 15.3. First we note that (15.42) is trivially 
satisfied for any choice of z if x =a, m, or b. Therefore, assume x #a,x #m,x#¥b, 
keep x fixed, and introduce a new function F defined on [a, b] by the equation 


F(t) = AMISH — PO) — AOL) — PO), 


A(t) = (t — a)(t — m)*(t — BD). 


where 


Note that F(t) = 0 for t = a, m, b, and x. By Rolle’s theorem, F’(t) vanishes in each of 
the three open intervals determined by these four points. In addition, F’(m) = 0 because 
A'(m) = 0 and f'(m) = P'(m). Therefore F’(t) = 0 for at least four distinct points in 
(a, b). By Rolle’s theorem F"(t) = 0 for at least three points, F(t) = 0 for at least two 
points, and F(t) = 0 for at least one point, say for ¢ = z. From the definition of F we 
find 


FOt) = AD)FOD — PEO] — APO) — POD] 
= A(X) F(t) — 4! LF) — PQ)). 
When we substitute ¢ = z in this equation we obtain (15.42). 


Now it is a simple matter to prove Simpson’s rule in the following form. 


THEOREM 15.14. SIMPSON’S RULE. Assume f has a continuous fourth derivative on 
[a, b], and let m = (a + b)/2. Then we have 


b—a _ (b= a) pw 
2 [f(a) + 4f(m) + f(d)] ae80 fc) 


(15.43) [ f00 dx = 


for some c in [a, b}. 
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Proof. Let M, and m, denote, respectively, the maximum and minimum values of 

f™ on [a, b], and let B(x) = —(x — a)(x — m)?(x — b)/4!. Since B(x) > 0 for each x 
in [a, b], Equation (15.42) leads to the inequalities 


Integrating, we find 


(15.44) m, | B(x) dx < | (P(x) — fo) dx < My [” Ba) dx. 


To evaluate the integral [° B(x) dx we let h = (b — a)/2 and we have 
b 1 b 1 h 
| B(x) dx = -i | (x — a)(x — m)?(x — b) dx = -i | (t + h)t*(t — h) dt 
a 7 Ja 2 J—h 


h __ NS 
--i[#e- ae ee, a 
4! Jo 4! 15 2880 
Therefore the inequalities in (15.44) give us 


— 


mS zeal [P(x) — f(x] dx < My. 
But since f) is continuous on [a, 5], it assumes every value between its minimum m, and 
its maximum M, somewhere in [a, b]. Therefore 


7%) = ae 


ay | Pe — soo ax 
for some c in [a,b]. Since [° P(x) dx = $(b — a)[f(a) + 4f(m) + f(5)], this equation 
gives us (15.43). 


Simpson’s rule is of special interest because its accuracy is greater than might be expected 
from a knowledge of the function fat only three points. If the values of fare known at an 
odd number of equally spaced points, say ata,a +h,...,a-+ 2nh, it is usually simpler 
to apply Simpson’s rule successively to each of the intervals [a, a + 2h], [a + 2h, a + 4h], 

, rather than to use an interpolating polynomial of degree <2n over the full interval 
[a,a + 2nh]. Applying Simpson’s rule in this manner, we obtain the following extension 
of Theorem 15.14. 


THEOREM 15.15. EXTENDED SIMPSON’S RULE. Assume f has a continuous fourth derivative 
in [a,b]. Let h = (b — a)/(2n) and let f, = f(a + kh) fork =1,2,...,2n—1. Then 
we have 


) “f(x) d 


(F(a) + 4 Sanat 2> Sut JO) - - Ca pW) 


for some € in [a, b). 


The proof of this theorem is requested in Exercise 9 of the next section. 
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15.21 Exercises 
1. (a) Apply the trapezoidal rule with n = 10 to estimate the value of the integral 


ee 2 dx 
og a re 


Obtain upper and lower bounds for the error. (See Exercise 10(b) to compare the accuracy 
with that obtained from Simpson’s rule.) 
(b) What is the smallest value of that would ensure six-place accuracy in the calculation of 
log 2 by this method? 

2. (a) Show that there is a positive number c in the interval [0, 1] such that the formula 


[* fed dx =f +f(-0 


is exact for all polynomials of degree <3. 
(b) Generalize the result of part (a) for an arbitrary interval. That is, show that constants c, 
and c, exist in [a, b] such that the formula 


b b-—a 
| f(x) dx = aaa [f(cy) + f(co)] 


is exact for all polynomials of degree <3. Express c, and c, in terms of a and b. 
3. (a) Show that a positive constant c exists such that the formula 


1/2 
—1/2 


f(x) dx = $31f(-—c) +f) + fo) 


is exact for all polynomials of degree <3. 
(b) Generalize the result of part (a) for an arbitrary interval. That is, show that constants c, 
and c, exist in [a, b] such that the formula 


ic dx = -=*| feo +4(2) + fed] 


is exact for all polynomials of degree <3. Express c, and c, in terms of a and b. 
4. Show that positive constants a and b exist such that the formula 


[, eG) dx = Hef) + FO) 


is exact for all polynomials of degree <3. 
5. Show that a positive constant c exists such that the formula 


| e*"f (x) dx = ve [f(—c) + 4f0) + fo) 


is exact for all polynomials of degree <5. 
6. Let P,, be the interpolation polynomial of degree <n that agrees with fat + 1 distinct points 


Xo» X15-++9Xm- 
(a) Show that constants A(x), A,(n),...,An(m) exist, depending only on the numbers 
Xo, X1,+++5%Xn, a, and b, and not on f, such that 


[? Pa(o) de = & Axl) flo). 
: =0 


The numbers A;(n) are called weights. (They are sometimes called Christoffel numbers.) 
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(b) For a given set of distinct interpolation points and a given interval [a, b], let W(n), 
W,(n),..., Wy(n) be n + 1 constants such that the formula 


[” fo) dx = > Wd fey) 
k=0 


is exact for all polynomials of degree <n. Prove that 


n 


an — q’ti 
2, ¥il0) = for aa | ed Roreeee / 3F 


This is a system of n + 1 linear equations that can be used to determine the weights. It can 
be shown that this system always has a unique solution. It can also be shown that for a suitable 
choice of interpolation points it is possible to make all the weights equal. When the weights 
are all equal the integration formula is called a Chebyshev integration formula. Exercises 2 
and 3 give examples of Chebyshev integration formulas. The next exercise shows that for a 
proper choice of interpolation points the resulting integration formula is exact for all poly- 
nomials of degree <2n + 1. 

. In this exercise you may use properties of the Legendre polynomials stated in Sections 6.19 
and 6.20. Let x9, x,,..., Xn be the zeros of the Legendre polynomial P,,,,(x). These zeros 
are distinct and they all lie in the interval [—1, 1]. Let f(x) be any polynomial in x of degree 
<2n +1. Divide f(x) by P,,,(x) and write 


F(®) = Prys(x)Q(x) + RO), 


where the polynomials Q and R have degree <n. 
(a) Show that the polynomial R agrees with fat the zeros of P,,, and that 


[' f@ dx = ie R(x) dx. 


(b) Show that n + 1 weights W,(n),..., W,,(n) exist (independent of f) such that 


[/,f0) dx = ¥ Wl f(xy). 


This gives an integration formula with n + 1 interpolation points that is exact for all poly- 
nomials of degree <2n + 1. 
(c) Take n = 2 and show that the formula in part (b) becomes 


[2 f@ ax = § f(—./3) + $ f(0) + 5 f(./8). 


This is exact for all polynomials of degree <5. 

(d) Introduce a suitable linear transformation and rewrite the formula in part (c) for an 
arbitrary interval [a, 5]. 

. This exercise describes a method of Peano for deriving the error formula in Simpson’s rule. 
(a) Use integration by parts repeatedly to deduce the relation 


{ u(t)v"(t) dt = u(t)v"(t) — u'(t)u'(t) + u"(Dw(t) — { g(t) dt, 


where g(t) = u"(t)v(t). 
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(b) Assume ¢ has a continuous fourth derivative in the interval [—1, 1]. Take 
v(t) =t(1 —1)?/6, = u(t) = g(t) + 9-2), 


and use part (a) to show that 
c y(t) dt = 319(—1) + 49(0) + g(1)] — [e@ae. 


Then show that J} g(t) dt = o'*)(c)/90 for some c in [—1, 1]. 
(c) Introduce a suitable linear transformation to deduce Theorem 15.14 from the result of 
part (b). 

9. (a) Let a,,a,,..., a, be nonnegative numbers whose sum is 1. Assume ¢ is continuous on 
an interval [a, 5). If cy, cg,..., C, are any n points in [a, b] (not necessarily distinct), prove 
that there is at least one point c in [a, 5] such that 


> a,9(cr) = 90). 
k=1 


[Hint: Let M and mm denote the maximum and minimum of ¢ on [a, 5] and use 
the inequality m < 9(c,) < M.] 


(b) Use part (a) and Theorem 15.14 to derive the extended form of Simpson’s rule given in 
Theorem 15.15. 

10. Compute log 2 from the formula log 2 = J? x! dx by using the extension of Simpson’s rule 
with (a)n = 2; (b)m =5. Give upper and lower bounds for the error in each case. 

11. (a) Let p(t) be a polynomial in t of degree <3. Express g(t) by Newton’s interpolation 
formula and integrate to deduce the formula 


f° p(t) dt = $[~(0) + 3g(1) + 3y(2) + @)). 


(b) Let P be the interpolation polynomial of degree <3 that agrees with f at the points a, 
a+h,a+2h,a + 3h, where h > 0. Use part (a) to prove that 


a+3h 3h 
{ P(x) dx = 3 If@ + 3f(a + h) + 3f(a + 2h) + f(a + 3h)). 


(c) Assume f has a continuous fourth derivative in [a, b], and let h = (6 — a)/3. Prove that 


(6 — a) 
6480 


b és 
{ f(x) dx = — [f(a) + 3f(a +h) + 3f(a + 2h) + f(b) - IO 


for some c in [a, 6]. This approximate integration formula is called Cotes’ rule. 
(d) Use Cotes’ rule to compute log 2 = J? x" dx and give upper and lower bounds for the 
error. 

12. (a) Use the vector equation r(t) = asint i + bcostj, where 0 <b <a, to show that the 
circumference L of an ellipse is given by the integral 


1/2 
L = 4a |" 1 — sin? ¢dt, 


where k = ,/a® — b/a. 
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(b) Show that Simpson’s rule gives the formula 
5 
ores 24 Fay) — fla) 
L=,; [a+b + ./8(a? + 5°)] >a040 4 (c) 
for some c in [0, 7/2], where f(t) = J1 — k*sin? t. 


15.22 The Euler summation formula 


Let n be a positive integer. When the trapezoidal formula (Theorem 15.13) is applied to 
the interval [0, n] it becomes 


n _& ; fon 
| f(x) dx => fll) + F(a) = FO) — EM" 


for some c in [0,n]. If fis a quadratic polynomial then f” is constant and hence f"(c) = 
f’(0). In this case the formula can be rewritten in the form 


(15.45) > fo = [409 ae +O ein +o" 


It is exact when fis any polynomial of degree <2. 

Euler discovered a remarkable extension of this formula that is exact for any function 
with a continuous first derivative. It can be used to approximate integrals by sums, or, as 
is more often the case, to evaluate or estimate sums in terms of integrals. For this reason 
it is usually referred to as a “summation’”’ formula rather than an integration formula. It 
can be stated as follows. 


THEOREM 15.16. EULER’S SUMMATION FORMULA. Assume f has a continuous derivative on 
[0, n]. Then we have 


15.46) > f(y = [pt dx ORL 4 x — Ed —pyreas, 


k=0 


where [x] denotes the greatest integer <x. 


Proof. Integration by parts gives us 


(15.47) [" & — DF) dx = (n — DCm) + 40) — |” FOO a. 


Now we consider the integral [” [x] f'(x) dx and write it as a sum of integrals in each of 
which [x] has a fixed value. Thus, we have 


IP bese) dx => J pay dx =r JF) ae 


= nse + 1)-—f(r)) = 2 fr + 1) — > f(r) 


-T/F+ N+ TOF V+ Y-TIO 


— ¥ f(k) + nfm) = - SI) + $O) + nf. 
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Subtracting this from Equation (15.47) we obtain 


|, "(x — ke] — Df'Qo dx => 10 a = i) " f(x) dx, 


which is equivalent to (15.46). 


The last integral on the right of (15.46) can be written as 


[= bl — pre ax = |” poos'co ax, 


where q, is the function defined by 


x—[x])— 3 if xis not an integer, 
P(x) = 
0 if x is an integer. 
The graph of ¢, is shown in Figure 15.6(a). We note that g,(x + 1) = (x), which means 
that g, is periodic with period 1. Also, if O0<x<1 we have 9g,(x)=x-—4, so 
fi v(t) dt = 0. 
Figure 15.6(b) shows the graph of @., the indefinite integral of p,, given by 


G2(x) = i p(t) at. 
It is easily verified that qm, is also periodic with period 1. Moreover, we have 


ae | 
p(x) = = YG ree St, 
This shows that —4 < (x) < 0 for all x. The strict inequalities —} < (x) < 0 hold 
except when x is an integer or half an integer. 

The next theorem describes another version of Euler’s summation formula in terms of 
the function g,. 


P(x) P(x) = [e(oae 


FiGurE 15.6 Graphs of the periodic functions ¢, and 9. 
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THEOREM 15.17. Jf f” is continuous on [0, n] we have 


0 


(15.48) : f(k) = i " f(x) dx Oem = { " palx)f "CO dx. 
Proof. Since g(x) = 9,(x) at the points of continuity of @,, we have 


\, Pix)f (x) dx = i p(x) f'(x) dx. 


Integration by parts gives us 


[ po) dx = pAS'0O = [ gxCOF"@ dx = — |" gf") ax, 


since y(n) = (0) = 0. Using this in (15.46) we obtain (15.48). 


Note: Although Theorems 15.16 and 15.17 refer to the interval [0, 7], both formulas 
are valid when 0 is replaced throughout by | or by any positive integer < n. 


To illustrate the use of Euler’s summation formula we derive the following formula for 
log nt. 


THEOREM 15.18. For any positive integer n we have 
(15.49) log n! = (n+ 3) logn—n+C+ E(n), 
where 0 < E(n) < 1/(8n) and C = 1 + JP? t-*@,(t) dt. 


Proof. We take f(x) = log x and apply Theorem 15.17 to the interval [1,7]. This 
gives us 


> log k -| logx dx + 4logn + | PAD ay, 
t=1 1 1 XxX 
Using the relation f log t dt = t log t — t we rewrite this equation as follows, 
(15.50) logn! =(n+4)logn—n+1+4+ | PAD oy, 
/1 


Since |.2(t)| <4 the improper integral {? t-*y,(t) dt converges absolutely and we can 


write 
: P2(t) dt = E. (2(t) dt a [° P(t) dt. 
1 2 ae t” 


n 


Therefore Equation (15.50) becomes 


log n! = (n+ #)logn—n + ~ { Pe at, 


n 
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where C = | + JP t-*@,(t) dt. Since we have —4 < 9,(t) < 0 except when t¢ is an integer 


or half an integer, we obtain 
0< -| PMO act, 
n ft 8n 


This proves (15.49), with E(n) = —J® t-*@,(t) dt. 


From Theorem 15.18 we can derive Stirling’s formula for estimating n!. 


THEOREM 15.19. STIRLING’S FORMULA. If n is a positive integer we have 


2a nrtileo-n n! </20 nvnitern( +2). 
n 


Proof. Using Equation (15.49) and the inequalities for E(n) we obtain 


exp ((n + 4) logn —n+C) <n! < exp (+ Hlogn—n+c4e), 
n 


where exp (t) = e’. Using the relation e” < 1 + 2x, with x = 1/(8n), we can rewrite these 
inequalities as 


(15.51) An™**e—™ < nl < An™t'2e- (1 + rm 
n 


where A = e“. To complete the proof we need to show that A = J2n. 
We shall deduce this from the inequality 


(15.52) m< (ar) < 


m2n + 1) 
(2n)! 


2 


discovered by John Wallis (1616-1703). First we show how Wallis’ inequality implies 


A =~/27; then we discuss the proof of (15.52). 


If we let 
n! 
A, = ynrteg-n 
the inequality in (15.51) implies 


A<A,<A(147)). 
4n 


This shows that A, — A asn—> oo. In (15.52) we write n! = n"+/2e-"4_ to obtain 


Dye e =" AS 2 a(2n of 1) 
>" ere < ”) ’ 
which is equivalent to 
4 
A, ae 2n+ 1 . 


TT 
s 2A; 2n 
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We let n — oo in this last inequality. Since A = e© > 0 we obtain 
4 
T< ce <7. 
2A? 


This shows that A? = 277,so A = Fy Qn, as asserted. 
It remains to prove Wallis’ inequality (15.52). For this purpose we introduce the numbers 


1/2 
1,= | sin” t dt, 
0 


where n is any nonnegative integer. We note that J, = 7/2 and], =1. ForO<t< 7/2 
we have 0 < sint <1; hence 0 < sin”™'!t < sin” ¢. This shows that the sequence {I,} 
is monotonic decreasing. Therefore we can write 


I oak 1 


Ientan—1 7 i : Tentoans 


(15.53) 


Now we shall evaluate each member of this inequality; this will lead at once to Wallis’ 
inequality. 
Integration of the identity 


< (cos tsin"** 1) =(n + 1)sin"t — (n + 2)sin™?t 


over the interval [0, 7/2] gives us 


0= (n a I)l, _ (n a 2). 
or 


n+1 
15.54 D5 I 
( ) re 


Using this recursion formula with n replaced by 2k — 2 we find 


I, 2k—1 2k(2k —1) 


Tong = 2k ~~ (2k)? 


Multiplying these equations for k = 1, 2,...,m, we find 


Nt ae i (2k)? 2?"(n!)? 


The product on the left telescopes to I;,,/Ip. Since Ig = 7/2 we obtain 


(15.55) Ion = 
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In a similar way we apply the recursion formula (15.54) with n replaced by 2k — 1 and 
multiply the resulting equations fork = 1, 2,...,n to obtain 


TT Toit _ % (2k)? 2°"(n!)° 1 a7 1 


k=-1I5, , e=1 2k(2k + 1)  (Qn+1)! 2n+1 2 In, 
The product on the left telescopes to Jo,41/J, = Jeny1, SO we get 


T 


15.56 |e Pee ee 
( ) andanta = 2057 yd) 


Since [g,41 = 2nlg,_,/(2n + 1), Equation (15.56) implies 


vie 
Ientlon—a Si 


We use this in (15.53), together with the two relations (15.55) and (15.56). Then we multiply 
by 7/4 to obtain Wallis’ inequality (15.52). 


15.23 Exercises 


1. If fis a polynomial of degree < 2, show that Euler’s summation formula (15.48) reduces to the 
trapezoidal formula, as expressed in Equation (15.45). 
2. Euler’s constant C is defined by the limit formula 


— 
C = tim ( ; - toe), 


noo k=1 
(See Section 10.17 in Volume I.) Use Euler’s summation formula to prove that 


E(n) 
=—log nC +o = 


: 2n n 


k=1 


where 0 < E(n) < }. Also, show that 
orf—t{t 
C=1 -{ “ dt 
f 
1 


3. (a) If s > 0, s #1, use Euler’s summation formula to prove that 


25 = +045] aes 
kK pst : 


1 or — [t] 
C(s) = 1 tons] a. 


where 
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(b) If s > 1, show that C(s) = f(s), where ¢ is the Riemann zeta function defined for s > 1 
by the series 
— 
C(s) = > ie 
k=1 


The series for ¢(s) diverges for s < 1. However, since the formula for C(s) in part (a) is meaning- 
ful forO <s < 1, it can be used to extend the definition of ¢(s) to the open intervalO <s <1. 
Thus, for s >Oands # 1 we have the formula 


= 1 vi-td, 
f(s) = ze ee aa Pepe ees 


This is a theorem if s > 1, and a definition if0 <s <1. 


In Exercises 4 through 6, 9, is the function introduced in Section 15.22. 


4. (a) Use Euler’s summation formula to prove that 
a log x — 
S log? = (n + $) log’ n — 2nlogn + 2n —2 +2 “pata B= dx. 
k=1 


(b) Use part (a) to deduce that for n > e we have 
n 
> log? k = (n + 4) log? n — 2nlogn + 2n + A — E(n), 
k=1 


logan 
4n 
5. (a) Use Euler’s summation formula to prove that 


where A is a constant and 0 < E(n) < 


—logk 1 ; llogn "2logx — 
> k = Slogtn + 528" — | Beg P(x) dx. 


k=1 


(b) Use part (a) to deduce that for n > e” we have 
“,logk 1 llogn 
= = = log? n +5— +A — E(n), 


log n 
8n? 
6. (a) If > 2 use Euler’s summation formula to prove that 


S 1 
— k log k 


= log (og n) + ———— 


where A is a constant and 0 < E(n) < —> 


2+3logx +2 log’ x 
A NETS XxX 


— log (log 2) — [ Po(x) (x log x) 


1 1 
2n log n 14 log 2 
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(b) Use part (a) to deduce that for n > 2 we have 


— 1 1 
2 Tlogk = 18 0B") +4 +57 — FW, 


where A is a constant and 0 < E(n) < —,-—_.. 
4n* log n 


7. (a) Ifa > O and p > 0, use Euler’s summation formula to prove that 


1 oO 
+ 5 | p,(x)x? te" dx, 
0 


where I is the gamma function. 
(b) Use part (a) to deduce that 


: r(1+-) 
yew =-4,* ee ae ae Cee ee 


k=0 


8. Deduce the following limit relations with the aid of Stirling’s formula and/or Wallis’ inequality. 
(a) Tim Ga We ANE = 
(n!)222" = 


(b) lim ota V™ 


(c) lim ( v7) : 
Cc aa n aaa 


7/2 
9. Let I, = | sin” ¢ dt, where n is a nonnegative integer. In Section 15.22 it was shown that the 


0 
sequence {J,,} satisfies the recursion formula 


n+1 


In+e =o 


— _[n+1 n ; ; 
Let f(n) =4 Ja r( \/r(5 + 7 , where I’ is the gamma function. 


(a) Use the functional equation I'(s + 1) = sI(s) to show that 


f@+2)=—,; 5 fen (1). 
(b) Use part (a) to deduce that 


= 
+ 


sin” tdt = -— 


ae 
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1.10 Exercises (page 13) 


1. Yes; 2 5. Yes; 1 9. Yes; 1 13. Yes; n 

2. Yes; 2 6. No 10. Yes; 1 14. Yes; n 

3. Yes; 2 7. No 11. Yes; an 15. Yes; n 

4. Yes; 2 8. No 12. Yes; n 16. Yes; n 
17. Yes; dim =1+4n ifniseven, 3(n +1) if nis odd 


1 

18. Yes; dim =n ifniseven, 4(7 +1) if nis odd 

19. Yes; kK+1 

20. No 

21. (a) dim =3 (b) dim = 3 (c) dim =2 (d) dim =2 

23. (a) Ifa #0and b #0, set is independent, dim = 3; if one of a or b is zero, set is de- 
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(i) Independent; dim = 2 (j) Independent; dim = 2 


1.13 Exercises (page 20) 
1. (a) No (b) No (c) No (d) No (e) Yes 


2 


8. (a) 4/e2? +1  (b) g(x) = 6(x -5 


(n + 1)Qn + 1) n+1 
6n ae: 
11. (c) 43 (d) g(t) =a(l — §t), a arbitrary 
12. (a) No (b) No (c) No (d) No 
13. (c) 1 (d) e? —1 

14. (c) n!/2ntl 
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1 
4 3 6 arbitrary 


2n + 1 


10. (b) b (c) gi) =a i ), a arbitrary 
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1.17 Exercises (page 30) 


~ 
rN 


Ce OY ae eee 


(a) and (b) $./3,1,1), $/6(,-2,1) | 
(a) $/2(1,1,0,0), 4/6 (—1,1,2,0), 4/3 (1, -1, 1, 3) 


= 1 
(b) 4/3 C1, 1, 0, 1), Ja (1, —2, 6, 1) 


& — 4 log®3 9. 2 —2sinx 

e= — | 10. # —4x 

4(e — e) ox; 1 — Te? 
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Linear; nullity 1, rank infinite 26. Linear; nullity infinite, rank 2 


Linear; nullity 2, rank infinite 
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c = wand f(x) = c, cos x + c,sin x, where c,, Cz are not both zero but otherwise arbitrary ; 
ifc, #0, thenc = 27 and f(x) = c,, where c, is nonzero but otherwise arbitrary. 


Exercises (page 42) 
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No 14. No 
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Yes; =u, y=v-u, Z=w-v 
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(S + T)® = S? + TS? + STS + S*T + ST? + TST + T?S + T? 


nS 
pee 
MOOR 
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26. (a) (ST)(x,y,z) =(x ty +2z,x +y,x); (TS), y,2) =(Z,z+y,z7+y +x); 
(ST — TS)(x, y,z) =(x +y,x —2z, -—y —2z); S*(x, y,z) = (x,y,z); 
T*(x, y,Z) = (x, 2x + y, 3x +2y +2); 
(ST)*(x, y, z) = (3x +2y +2,2x +2y +z,x +y +2); 
(TS)*(x, y, 2) = (x +y +2,x + 2y + 22z,x + 2y 4+ 32); 
(ST — TS)? = (2x +y —2,x +2y +2, —x +y + 22) 
(b) S (u,v, w) =(w,v,u); T (u,v, w) = (uyv —u,w — dv); 
(ST) (u, v, w) = (wiv —w,u—v); (TS) (u,v, w) = (w — 9,0 — U, WU) 
(c) (T-—Dx, y,z) =(,x,x +y); (T-—D*(x, y, z) = (0,0, x); 
(T — D(x, y,z) =(0,0,0) if n>3 
28. (a) Dp(x) =3 — 2x + 12x?; Tp(x) = 3x — 2x* + 12x32; (DT )p(x) = 3 — 4x + 36x; 
(TD)p(x) = —2x + 24x?; (DT — TD)p(x) = 3 — 2x + 12x*; 
(T?D? — D®T*)p(x) = 8 — 192x (b) p(x) = ax, aan arbitrary scalar 
(c) p(x) = ax® + b, aand b arbitrary scalars (d) AllpinV 
31. (a) Rp(x) =2; Sp(x) =3 —x +x7; Tp(x) = 2x + 3x2 — x8 4+ x3; 
(ST)p(x) =2 +3x — x 4+ x3; (TS)p(x) = 3x — x7 +-.x3; (TS)*p(x) = 3x — x + 3; 
(T?S*)p(x) = —x2 +x°5 (S?T*)p(x) = 2 + 3x — x? +.x3;5 (TRS)p(x) = 3x; 
(RST)p(x) = 2 (b) N(R) ={p| p(0) =0}; R(V)={p|pisconstant}; N(S) = 
{p|pisconstant}; S(V)=V; N(T)={O}; T(V) ={p| pO) =0} (c) ToO=S 
(d) (TS" =1—R; S"T™=1 
32. T is not one-to-one on V because it maps all constant sequences onto the same sequence 


2.12 Exercises (page 50) 


1. (a) The identity matrix J = (6,,), where 6,, = 1ifj =k, and 6, =Oifj Ak 
(b) The zero matrix O = (a,,) where each entry a,, = 0 
(c) The matrix (cd,,), where (6;,) is the identity matrix of part (a) 


010 0 0 


1 0 0 0 1 0 
2. © | © | | (c) 10 0 1 0 0 
0 1 0 00 1 


0001 0 
3. (a) —-Si+ 7, 9 — 12) 


1 2 3 0 = 3 0 
OL abba @ Ly ab teal 
14 0 3 1 2 0 3 
—2 0 4 0 
Doak lea 
0 2 0 4 


5. (a) 384+ 47 + 4k; nullity 0, rank 3 (b) 1 -—3 3 


jn 


ON 
—" 
| 
ary 
ry 


10. 


11. 


12. 


13. 


14. 


15. 


16. 


17. 


18. 


(a) 


(c) 


(a) 


(a) 


(c) 
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0 1 1 
T(4i —j +k) = (0, —2); nullity 1, rank 2 (b) 
0 1 —-1 


0 1 3 
(d) aT =f, e, =k, es =i, w, = (1, 1), w, = (I, —1) 
0 0 —2 


1 —1 
(5,0, —1); nullity 0, rank 2 (b) | 0 0 
1 1 


e.=i, e@ =itf, w,=(1,0,1), mw, =(0,0,2), ws = (0, 1,0) 
1 1 

(—1, —3, —1); nullity0O,rank2 (b) [0 1 
1 1 


=i, 2) =j—i, w, = (1,0, 1), we = (0, 1,0), wz = (0, 0, 1) 


1 2 
é; — @,; nullity 0, rank 2 (b) | 4 (c) a=5, b=4 


de A 


o Oo & 
_~ Oo © 


-1 1 0 -1 0 0 -2 
0 1 0-1 2 O 
0 -1|° 0 0-1 O 

0 1 Oo 0 0 oO -1 
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0 00 0 0 1 0 0 000 0 
0 1 0 0 00 4 0 0 02 0 
19. (a) (b) (c) 
0 02 0 0 0 0 9 0 0 0 6 
0 0 0 3 0 0 0 0 0 0 0 0 
0 —1 0 0 0 0 0 0 0 —8 0 
0 0 —2 0 0 1 0 0 0 0 0 —48 
(d) (e) (f) 
0 0 0 -—3 0 0 4 0 0 0 0 0 
0 0 0 0 00 0 9 0 0 0 0 


20. Choose (x°, x?, x, 1) as a basis for V, and (x?, x) as a basis for W. Then the matrix of TD is 
6 0 0 0 
020 0 


2.16 Exercises (page 57) 


3. 4 -~1 #4 -2 
15 —14 
1 B+C=!|/0 2|, AB= _ BA=|-4 16 8], 
6 —5 ae as 7 —28 14 
0 oO oO 
0 0 30 —28 
AC = , CA=}2 -8 4)/, AQB-3C)= 
0 0 4 16 —30 28 


a b —2a a 
2. (a) x ale a and b arbitrary (b) al a and 6 arbitrary 


2b 
3. (a) a=9, b=6, c=1, d=5 (b) a=1, b=6, c=0, d=-2 
-9 -—2 -10 -3 5 —4 
4. (a) 6 14 8 (b) 0 3 24 
—5 12 -27 0 


evel 
0 1 


° nO —sin 7 


:. nO cosné 


“ae + 1) 


10. 


11. 


12. 


14. 
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1 0 
—100 1 
a b 
, where b and ¢ are arbitrary, and a is any solution of the equation a? = —bc 
c¢ —a 


a 0 
(b) , where a is arbitrary 
0a 


1 0 —1 
0 1}' | 0 
of the equation a? = 1 — be 
yoy 2 
8 7 43 25 


(b) (A + B)? = A? + AB + BA + B?; 
(c) For those which commute 


0 a 6b 
’ , and , where band care arbitrary and ais any solution 
= c —a 


(A + B)(A — B) = A? + BA — AB — B? 


2.20 Exercises (page 67) 


13. 


16. 


SS al 


(x,y, z) = G, —$, 3) 4. (x, y, z) = (1, —1, 0) + i =3, 4,1) 

No solution 5S. (x, y,z,u) = (1,1, 0,0) + ¢(1, 14, 5, 0) 
(x, y,z) = (1, —1, 0) + ¢(—3, 4, 1) 6. (x, y,z,u) = (1, 8,0, —4) + ¢Q, 7, 3, 0) 
(x, y,Z, u,v) =t,(—1, 1, 0, 0,0) + t.(—1, 0, 3, —3, 1) 

(x, y,z,u) = (1,1, 1, —1) +4,(—1, 3, 7, 0) + 4(4, 9, 0, 7) 

(x, y,z) = (3,2, 0) + 4(5, 1, —3) 

(a) (x, y,z,u) = (1, 6, 3,0) + 4,(4, 11, 7, 0) + 4(0, 0, 0, 1) 

(b) (x,y,z, u) = (3, 4, 72, 0) + ¢(4, —11, 7, 22) 


=f 2 4 14 8 3 
5: aR: eG 14.18 5 2 
3 5 4 321 
P22 oF © 
-$ § #3 
0 1-2 1 
-1 oO 1 15. 
0 0 —2 
3 -3 -8 
0 0 1 
0 2+ oO-1 0 1 
1 0 0 0 0 =O 
0 0 0 1 O -1 
3 0 1 0 0 =O 
0 0 0 0 0 
9 0-3 0 1 0 
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2.21 Miscellaneous exercises on matrices (page 68) 


2 1 
3, P= 
5 -l 


0 0 1 0 a b 
é , and , where 6 and ¢c are arbitrary and a is any solution 
0 0 0 1 c l-a 


of the quadratic equation a*#—a+bc =0 
1 1 1 1 —1 1 1 —-1 —1 —-1 —1 —1 
10. (a) ’ 9 ’ ’ 9 ’ 
—1 1 1 -1 1 1 1 1 1 —-1 —1 1 
1 -1 —1 1 
a ay Wey 2d 
3.6 Exercises (page 79) 


(a) 6 (b) 76 (c) a® —4a 

(a) 1 (b) 1 (c) 1 

(b) (6 —al(c — alc — ba +b +c) and (b — alc — alc — b\ab + ac + be) 
(a) 8 (b) (6 —aje — ad — ajc — b)(d — b)\d — c) 

(c) (b —a)(c —a)d —a)(c — b)d — b)(d —c)(a+b+c+d) 

(d) a(a® — 4)(a® — 16) (e) —160 


fife fs} jf hk fst |i h fe 


7. F'=!81 82 &3|+/8& &2 &3) +18 &2 2&3 
Ay Ne he Ay ftp Ng hi hi i 


Chapter 3 


ela 


fi fp fs fh ft fs 
8. (b) F=(fi fo fz| then F’=/f, fo fs 
i fz fs 1 fe fe 
1 _3 1 pe Se 
2 4 8 16 
i , {9 #& -# 3 
10 det A =16, det (4 )= 76? A= 0 0 4 —3 
0 0 0 4 


3.11 Exercises (page 85) 


6. det A = (det B)(det D) 
7. (a) Independent (b) Independent (c) Dependent 


3.17 Exercises (page 94) 
109 113 -—41 —13 


4 -3 —40 —92 74 16 

l. (a b) | —6 3 3 Cc 
= au) | ” —41 —79 7 47 
—50 38 16 20 


Answers to exercises 


109 —40 
2 -6 —4 
if 4 —2 1 ; | 113 -92 
—- b) -| -1 3 —2 — 
| ee ©) 306] 41 74 
° : —13 16 
3. (a) A=2, A= —-3 (b) A=0, A= +3 (c) A=3, A= ti 
5. (a) x =0, y= l, =2 (bt) x=1, y=l, z=-1 
x y zl 
xX ~X YT) 2% 7 244 
xi yi 21 1 
6. (b) det|%2—*%1 Ye—yi 22 —2%1| =0; det ~0 
X_ yo Z | 
X3—%, 3 43 — 41 
X3 y3 23 1 
(x — x1)? + (y — yy)? (x — x) (y — yy) 
(c) det (xp — x)? + (v2 — yy)? (x2 — X,) (Ye — yi) = 0; 
(x3 — x)? + (V3 — y,)? (x3 — %) (ys — yx) 
xe+y> x y 1 
ety x, yy 1 
det , : = 0 
Xe tye Xe ye 1 
xet+ys Xs ys | 


Chapter 4 
Exercises (page 101) 


5. Eigenfunctions: f(t) = Ct*, where C ¥ 0 

6. The nonzero constant polynomials 

7. Eigenfunctions: f(t) = Ce*/*, where C # 0 
8. Eigenfunctions: f(t) = Ce’#’/4, where C # 0 


10. Eigenvectors belonging to 4 = 0 are all constant sequences with limit a #0. Eigenvectors 
belonging to 4 = —1 are all nonconstant sequences with limit a = 0 
4.8 Exercises (page 107) 
Eigenvalue Eigenvectors dim E(A) 
1. (a) 1,1 (a, b) ¥ (0, 0) 2 
(b) 1,1 t(1,0),¢ #0 1 
(c) 1,1 1(0,1),¢ #0 1 
(d) 2 t(1,1) ¢ #0 1 
0 t(1, —1),¢ #0 1 
2 1 + fab t(./a, /b), t 0 1 


t(/a, —/b), t #0 1 


—50 
38 
16 
20 


629 
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3. If the field of scalars is the set of real numbers R, then real eigenvalues exist only when 
sin 6 = 0, in which case there are two equal eigenvalues, 4, = 4, = cos 9, where cos 6 = 1 or 
—1. In this case every nonzero vector is an eigenvector, so dim E(A,) = dim E(A,) = 2. 

If the field of scalars is the set of complex numbers C, then the eigenvalues are 4, = 
cos 6 + isin 6, A, =cos 6 — isin @. Ifsin 6 = 0 these are real and equal. Ifsin 0 # 0 they are 
distinct complex conjugates; the eigenvectors belonging to 4, are t(i, 1), tf #0; those be- 
longing to A, are t(1, 1), #0; dim E(A,) = dim E(A,) = 1. 


ab 
4. , where b and c are arbitrary and a is any solution of the equation a? = 1 — be. 
c —a 


a b 
5. Let A = ,andlet A = (a — d)* + 4bc. The eigenvalues are real and distinct if A > 0, 
c d 


real and equal if A = 0, complex conjugates if A < 0. 


6. =b=c=d=e=f=1. 
Eigenvalue Eigenvectors dim E(A) 

7. (a) 1,1,1 (0,0, 1),¢ #0 J 
(b) 1 t(1, —1,0),¢ #0 1 
2 t(3,3, —1),¢ #0 1 
21 t(1,1,6),¢ #0 1 
(c) 1 t(3, —1,3),¢t #0 1 
Dig (2,2, —1),t #0 1 


8. 1,1, —1, —1 for each matrix 


4.10 Exercises (page 112) 


—2c 0 
2. (a) Eigenvalues 1,3; C = | where cd # 0 
c ad 


2a b 
(b) Eigenvalues 6, -—1; C= , Where ab #0 
Sa —b 


(c) Eigenvalues 3, 3; if a nonsingular C exists then C-'AC = 31, so AC = 3C, A = 31 
(d) Eigenvalues 1, 1; if a nonsingular C exists then C‘1AC =I,so AC =C,A =I 

3. -C=AB. 

4. (a) Eigenvalues 1,1, —1; eigenvectors (1,0, 1), (0, 1,0), 1,0, —1); 


1 0 1 
C=|0 1 0 
1 0 -!1 
(b) Eigenvalues 2, 2,1; eigenvectors (1,0, —1), (0,1, —1), (, —1, 1); 
1 0 1 
C= 0 1 -l 
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a b 
5. (a) Eigenvalues 2, 2; eigenvectors ¢(1,0), ¢ #0. If c=| | b #0, then 
—b 0 


2 0 
CTAC = 
1 2 


(b) Eigenvalues 3, 3; eigenvectors ¢(1,1), ¢4#0. If C -| 


3 0 
C-1AC = 
1 3 


6. Eigenvalues 1, 1, 1; eigenvectors (1, —1, —1),¢ #0 


a b 


» 56 £0, then 
a+b b 


Chapter 5 
5.5 Exercises (page 118) 
3. (b) TT” is Hermitian if n is even, skew-Hermitian if n is odd 


7. (a) Symmetric (b) Neither (c) Symmetric (d) Symmetric 
9. (d) Q(x + ty) = Q(x») + tQ(y) + HT(x), y) + (TQ), x) 


5.11 Exercises (page 124) 


1. (a) Symmetric and Hermitian 
(b) None of the four types 
(c) Skew-symmetric 
(d) Skew-symmetric and skew-Hermitian 


cos 0 sin 0 
4) | 


sin@ —cos 0 
5. Eigenvalues 4, = 0, 4, = 25; orthonormal eigenvectors u, = (4, —3), wu. = $(3, 4). 


6. Eigenvalues 4, = 2i, 4, = —2i; orthonormal eigenvectors 
oe —i), eee: c= i 
V2 V2 V2, -i i 
7. Eigenvalues 4, = 1, A, =3, A; = —4; orthonormal eigenvectors 


1 1 l 
“4, ==—— 1,0, 3), Ks = —— (3, 2, —1), u = —— (3, —5, —1). 
: Tio ‘ "14 ° 4/35 


fas | 3 3 
Vi0 Ji4 35 
2 —5 

eT) ° Jia 35 
3 -l -l 


vio Ji4 35 
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8. Eigenvalues 4, =1, A, = 6, 4; = —4; orthonormal eigenvectors 


1 1 
N= z(0, 4, —3), ky = 50 (5, 3, 4), Lk = T= (5, —3, —4). 
50 /50 


0 5 5 
1 2 
C=——| 4/2 3 -3 

Ym) *Y? 
—3,/2 4 -4 


9. (a), (b), (c) are unitary; (b), (c) are orthogonal 
11. (a) Eigenvalues 4, = ia, A, = —ia; orthonormal eigenvectors 


eid 1, i) ae y) (b) eae ae 
pee ae =f e 


5.15 Exercises (page 134) 


42 ! 
1. (a) ae[) : ®) 4-0 AS © mal), o 


ifi 1 
(i), eee. A) OS 
(c) wy a ), Us ha ) (d) Al; a 
1 4 
3. (a) a=] Pe (b) qi x2, dy = —4/2 
(c) m =10 + 2,1), wm =t(-1,1 + /2), wheres =1/N4 42,/2 


Paf2 <1 = 
(d) c= V i; where ¢ = 1/V4 + 2,/2 


1 8 1+,/2 
34 —12 
4. (a) A= (b) 4, =50, A, =25 
—12 41 
1 1 if 3 4 
(c) m4 =~-(3,-4), m==(4,3) (dd) CH=- 
? 5 ee 
14} 
5. (a) A=|% 0 3 (b) 4,=0, 4,=3, 4, = -} 
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(c) il 1, —1) (2, 1, 1) 0,1 1) 
Cc Wy = =U, 1, Ti), bg = Tah, 1,1), 4g = TW, 1, —- 
a i ep 
np 2 0 
1 = _ 
(c) C ~ 6 —./2 l J3 
—/2 1 —/3 
2 0 2 
6. (a) A=]|0 1 0 (b) A,=1, A, =3, A, = —2 
2 0 -1 
(c) (0, 1, 0) *_(2,0,1) (1,0 2) 
Cc uy = tn) » Wy = =,Y, » Us = =U, oe 
1 2 J 3 J 
0 2 1 
(dd) C= Cas . 5 0 0 
—2 
3 2 4 
7. (a) 2: QO: 2 (b) 4,=4,=-1, 4,=8 
4 2 3 
1 
(c) 4u aL 0,-1), mu =—~-(-1,4, —-1), = ~ (2, 1, 2) 
1 "He 2 3,/2 
—1 2./2 
@) C=—- 2 
33 0 4 v2 
—3 -1 2,/2 
8. Ellipse; center at (0, 0) 14. Ellipse; center at (0, 0) 
9. Hyperbola; center at (—3, —3) 15. Parabola; vertex at (2, ? 4 
10. Parabola; vertex at (33;, —{3 16. Ellipse; center at (—1, $ 
11. Ellipse; center at (0, 0) 17. Hyperbola; center at (0, 0) 
12. Ellipse; center at (6, —4) 18. Hyperbola; center at (—1, 2) 
13. Parabola; vertex at (#4, 44) 19. —14 
5.20 Exercises (page 141) 
8. a= +4/3 13. (a), (b), and (e) 
Chapter 6 
6.3 Exercises (page 144) 
1. = e8t — gt 6. (b) y =e! — ¢ 2/3 
2. y = Bx" + gx 7. y = cye?* + coe” 
3. y =4cosx — 2cos* x 8. y =¢, cos 2x + cg sin 2x 
4. Four times the initial amount 9. y =e%(c, COs 2x + cy sin 2x) 
5. f(x) =Cx", or f(x) = Cx” 10. y =e-*(cy + Cox) 
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ll. k =n?r*®; f(x) = Csinnax (n = 1,2, 3,...) 
13. (a) y’ -y=0 

(b) y’ —4y’ + 4y =0 

(c) yp’ +y +4y =0 

(d) y' +4y =0 

(e) y-y =0 _ 
14. y=4,/6, y" = —12y = —4,/6 


6.9 Exercises (page 154) 


l y=c, + ce + cge* 3. y=, + (cg + cgx)e 
2. y =c, + cge* + c3e* 4. yp = (cy + Cox + Cgx*)e* 
5. Va (c + Cox + Cax? + x*)e 

6. y =cye** + coe ** + ¢;cos 2x + c4 sin 2x 

1. y= e¥22(c, cos (2x + cg sin af 2x) + e-¥ 22(¢, cos,/2x + ¢,sin ,/2x) 
8. y = ce + e*/2(cy cos $4/3x + cy sin $4/3x) 

9. y =e *[(cy + Cox) COS x + (Cg + c4xX) Sin x] 
10. y = (cq, + cox) cos x + (cg + Cyx) sin x 
11, y = cy + 2x + (Cg + c4x) COS Jf 2x + (cs + Cgx) sin \/2x 
12, y=cy + yx + (cg + Cyx) COS 2x + (Cs + Cex) Sin 2x 


1 
13. f(x) = ane e™® — cos mx — sin mx) 


15. (a) y® —5y" + 4y =0 (ce) y) — ay) 4 y" = 0 
(b) y” + 6y” + 12y’ + 8y =0 (f) yt) + 8y” + 33y”" + 68y' + 52y =0 
(c) y) a 2y" + y" — 0 (g) y) _ 2y" + y = 0 


6.15 Exercises (page 166) 


1 yy = —2x — x? — 3° 5. yy = $x*e* + 2 
2. yy = yxe** 6. y, = $xe* 
3. yy =(&% — He" 7. yy =x cosh x 
4. y= + sin x 8 y= gax'e* 
9. 50y, = (11 — 5x)e* sin 2x + (2 — 10x)e* cos 2x 
10. yy = —(Bx + 8x? + yx )e™ 
0 x™exx 
a, a p(x) 


15. (b) 2D (c) 3D? (d) nD” 


6 wee epee ges | ae eg 
ee Aaa e e€ 5 © xX ad ae 


17. y =(A + $x)sin 2x + (B + } log |cos 2x|) cos 2x 
18. y = Ae* + Be* + 4secx 


19. y =(A + Bxje® + e% — xe* | ef dx + e* | xe®* dx 
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1 e4z 
+ ae | = dx + Ae* + Be®* + Ce™ 


6.16 Miscellaneous exercises on linear differential equations (page 167) 


1. u(x) = 6(e*” — e*)/5; v(x) =e* -—e™ 

2. u(x) = dessin 5x; v(x) = 3e-?*"7 sin 3x 

3. u(x) =e"; Ox) = 4242 

5. y =(A + Bx®je® + (x? — 2x + 2)e" 

6. y = Ae f ee 3 dy + Be 

7. y = Ax + Bx-* 

8. y = Ae* + Bx*e* — x 

9. y = A(x? — 2) + B/x 

10. y=x [A + Bx — 1)? + §x° + fx? — gx +4 -— (x — 18 log |x — 1] 
Hl. a=1,-1; y = [Ae?™ + Be-9)/x 


6.21 Exercises (page 177) 


2. f(x) = u(x) (« = 1) 
3. (a) A=(a—b)/2, B=(a+t+b)/2 


d 


d 
(b) i Gc — 1) 3 — a(x + 1)y =0, where «a = 1 or —2,andx = (¢ + 1)/2 


— —2)--+(a —2m +2 
i. esi > pe See 


m=1 


x?™ for all x; 


(a —1)(a — 3)-+- (a —2m +1) 
Se mn Siam OR a 


2m+1 
Qm +1! for all x 


Pees ee ae > (—1)"2m 
m=1 

5. u(x) =1 ae eee Te 

: 4 Gm + 2)3m — 1)---8-5 ‘ 


0 —] n 
U(x) = (1 + > one ) for all x #0 


n=1 
y= ai(z + 5 (@ = 2a — 3) @ an -1) 


nin +3)! 2") for all x 


n=1 


11. (b) f(x) = 5 Pox) + FPa(m) + a's Pa(x) 


2n 
15. (b) 


6.24 Exercises (page 188) 


2\2/cosx 
5. J_a(x) = -(— + sin x 


TX 
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9. (a) y=x- [eJi, ,(3x°?) + CoJ_1 (3x”*)] 
(b) y=x[e Ji, Gx) + CJ_1, (x *)] 
(c) y= x'8[e,J,(Qaxttm! 2) +4 CJ (Qaxttm/ *)], where « = 1/(m + 2), provided that 
1/(m + 2) is not an integer; otherwise replace the appropriate J by K 
(d) y =x'8[e,J,(4x2) + coJ_,(422)], where « = /2/8 
10. y =g, satisfies x2y” + (1 — 2c)xy’ + (a2b?x?® + c? — «*b*)y =0 
(a) y=x” [c,J5(2x4) + coK;(2x)] 
(b) y= 7” [c,J5.4(x) + ¢:J_54(x)] 
(Cc) y =x [oJ Gx) + cK, Gx)] 
(d) yr x[cyJo(2x”) + CK 9(2x)] 


ll. a=2, c=0 


[o.6) —1)" 
12. ye Sow: y =Jy(2x"2) ifx >0 


(n!)? 
=0 
13. b = (py — ap)/ag, ¢ = 4o/4 
14. y= x4 
~ n! 
= _ —_1)\nr-1 n 
IS. t=1: y =>( I Gy Gx) 
n=1 
(—1)” 
=e x% 1g 5-2/2 
(=: re (3) = xe 
16. u(x) =cosx; u(x) = 4 —tcosx — }cos2x 


Chapter 7 
7.4 Exercises (page 195) 


k—1 
35. (by - (PY = > PoP Pee 
m=0 
7.12 Exercises (page 205) 
l. (a) AM=21-—A, A* =nA-(n-1)I 


1 0 
(bt) e4 =e —tAI + te'A = “| 
t 1 
2. (a) At =31-4A4, A* = (2% —1A — 2" — 2) 


et 0 
(b) et4 = (2et — e®%)] + (e%* —e)A = 
gt pt git 


1+(-1)" 1-(-1)" 


cosht  sinht 
(b) ef4 = (coshr)/ + (sinht)A = 
sinht cosht 


10. 


11. 


13. 


7.15 


Oy ee 
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a ly 


a TE es 
(a) A A, A 5 5 


(b) ef! = (cosht)/ + (sinht)A = * i 
0 


e 
cos bt sin bt 

(b) eA = ett 
—sin bt cos bt 


0 e-1 
eA) 14 DAC): (ey = (@ DAH =| |: 
0 


0 
0 e 0 1 
0 0 0 0 


(a) A" =O if n>B3 


1 ¢ t+32 
1 
(b) et4 =] +4+1A Tora S 0 1 t 
0 0 1 


(a) A" =A if n>BIl 


1 
(b) et4 =] + (e' —1)A =|0 et ef — I 
0 


0 
(a) A® =4A* —5A 422; A" =|0 1 O 
1 


n 
e! 0 O 
(b) e4=|}0 e O 
0 te’ ef 


e4 =[ + 1A + 3t?A? 


ae e? —(e — 1) cat, e? (e — 1)? e? 0 
Ce = >; ervet = : eAtB = 
0 1 0 1 0 1 


Exercises (page 211) 

e'4 = 4(3e' — e®*)J + d(e** — eb)A 
1 
5 

eA = del{(t? — 2t + 2)7 + (—2t? + 20)A + t?A?} 
eA = (e-* — 3e-2t 4 e 3) ] ae (3e~ — de-2t 4 3 @-3t) 4 a (Je — e2%t 4 1e-3t) 4? 
etA = (det — 3e7! + 2te®)J + (4c?! — 3re?* — 4e')A + (e! — 7! + te?!) A? 
eta = (4e! = 6e2! 2c dest — ett)] ae (—72e! ae 1,9 o2t — Test af Jetty A 
+ (Bet — de + Ze3t — eft) 42 4 (—det + Let — Lest 4 Letty 43 
(b) eA = geW*{(6 — G6At + 3A7t? — A323) + (6t — GAL? + 3A7t3)A + (3? — 3At3) A? 4+ £343} 


e'4 = (cosh 5 aI + (sinh J5 t)A 
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= Cc, + 2c. = 2 2c, —C _ 
8. y, =c,cosh /5t + : = sinh /5 1, yz = Cg cosh ./5 t + - * sinh ./5 t 


J/5 
9. y, =e(cos 3t —sin3t), y, = e'(cos 3t — 3 sin 3¢) 

10. y, =e + 4te*, y, = —2e! + e% + 4te?*, ys = —2e! + 4e! 

11. yy =cye®, yo = ce’, yg = (Cot + cg)et 

12. y, =3e* —3e% +e, y. = —3e! + 6e°% — 3e 3, ys = Bet — 120%! + De! 
13. yy =e + Lae. Yo = 2e** — e784, ys = —e*! + et 

14. y, = —bet +e% + he, yo =e% +e%, ys =e, yy =e 
15. y, =2e* ~—1, yo =2e%* —t—2, ys =2e%*, y, = €?! 


At 


7.17 Exercises (page 215) 
Ya ee + 2(c + 1 — b)xe* 


2. (Cc) y= ee +2(c +1 — b)xe* +1, 
y, = —te' — get + he, y. = Ze! — get + de 
I 2 l m 
5. (a) By = 8B, B, =AB, B, = 5 AB... » Bn =A B 


(b) B= —m!(A7)"™41C 


1 1 3 | 1 
6 (a) Y(t)= (1 + ta +5 t°A® +a), where B = —6A7-4C = - | 


l 
This gives the particular solution y, = y») = —y3¢e — ¥st — Pyr? — 430 
3 
(b) y, =yo = —rde — vet — Yet? — He? + Tage 


7, E=B, F=-(AB+C) 


8. (a) yy = —cos2t—4}sin2t, y, = —$sin2¢r 

(b) y, =2cosh 2t + 3 sinh 2¢ — cos 2¢ —4sin2t, y, =cosh 2r + }sinh 2r — } sin 2¢ 
9. yy(x) = e?* + e®* — e®,  y.(x) = —2e% — e3* + 3e% 
10. y,(x) = seem — gee + (c, — goee* + (40 — Cy — Cy)xe*, 

yo(x) = gee” + see™™ + (cg — g00)e * + (cy + cy — 30)xe* 
Il. y,(x) = e**(2 cos x + sin x) + 26e7 — 77, yo(x) =e (sin x + 3cosx) — de" + 15 
12. yy(x) = e™(x? + 2x +3) +x? —3x +3, yo(x) =e *(-2x —2) 4 x, 

y3(x) = 2e% +x -1 


7.20 Exercises (page 221) 


4. (c) Y(x) = ete%AB 
5. If A(x) = > x*A,, then Y(x) =B+xC+ s x"B,, 
k=0 k= 
k 
where (kK + 2)(k + 1)By.2 = > A,B,, for k 20. 


r=0 


7.24 Exercises (page 230) 


1. (a) Y(x) =e 
Ae xk mA yk 
iby Vepee = > Tif misodd; Y,(x) = >» 7; if nis even 


k=0 k=0 
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x2 x x8 xi 
Ys0) = > +39 + 760 + 4400 
x! x? x10 


4 ' 14 160 
. x? 4x7 = 8x? i: 184xH . 4x18 
8) = 3 + 63 + 405 + 51075 * T2085 


3 
(a) Y.(x) =1+x+x7+— + 


xt Dx. Xe 
3 6 
(b) M=2; c=} 
4x3 7x4 = 6x® 
TO) eh a eee 


Y Vee x3 ss 2x° ‘: 17x? ‘ 38x29 " 134x112 ; 4x13 . xis 
(a 7? ae er aap 315 2835 51975 12285 59535 


4) ¥ ce od 62x? P se 
CO) TGA ee eh ge gag ggg ee ONL 


3x5 ene Zz 3x2 3x4 6x® 3x?) — 3x8 
20 Tio? OM =3x +7 +s tag + AH 


xt x6 Dx? x? 


¥x)HStxt+ ats + 3 + a5 3 


Perea eae 1ix® 29 7x 
se) -T FS te +o te tao tag 


(d) Y,(x) =0; lim Y,(x) =0 


Y3(x) =2 + 2x7 + x3 + 


n->C 

x? if x >0 x? if x>0 
(e) Y,,(x) = ; lim Y,(x) = 

—x? if x <0 now —x? if x< 
ok" 

() YG) = Zr 5 lim Yq) =0 

x? if x>0 x? if x>0 
(g) Y,(x) = > lim Y,(x%) = 
6 —-x? if x <0 noo —x* if x <0 


Chapter 8 
Exercises (page 245) 


All open except (d), (e), (h), and (j) 

All open except (d) 

(e) One example is the collection of all 2-balls B(O; 1/k), where A = 1, 2,3,... 

(a) Both (b) Both (c) Closed (d) Open (e) Closed (f) Neither 
(g) Closed (h) Neither (i) Closed (j) Closed (k) Neither (1) Closed 
(e) One example is the collection of all sets of the form S, = {x| |lxl| <1 — 1/k} for 
A =1,2,3,.... Their union is the open ball B(O; 1) 

No 
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8.5 Exercises (page 251) 


1. 


Ce 


11. 


E2; 


(a) 
(b) 


(j) 


All (x, y) 

All (x, y) ¥ (0, 0) 

All (x, y) with y #0 Be 

All (x, y) with y # 0 and — #5 +ka (k =0,1,2,. 
All (x, y) with x #0 uy 

All (x, y) # (0, 0) 

All (x, y) with xy # 1 

All (x, y) ¥ (0, 0) 

All (x, y) ¥ (0, 0) 

All (x, y) wih y #Oand0 <x <yory<x <0 


lim f(x, y) does not exist if x #0 


y—0 
(1 — m)/(1 + m?); No 
y =43x*; fnot continuous at (0, 0) 


f(,0) =1 


Exercises (page 255) 


f'xsy) sary 


(a) 
(b) 
(c) 


f'(x; y) =4 |x|? x-y 
All points on the line 2x + 3y = ts 
3z = 


All points on the plane x + 2y + 0 


fs y) =x° Ty) + y+ TO) 


Ox 


Ox 


Ox 
of 
Ox 


= y"/(x? + y?)” ; 


0 
= 2x + y* cos (xy) ; = = 2y sin (xy) + xy*® cos (xy) 


G) 
= x/Q? + "4; = = ylQ? + y?)4 


= —xy/(x? + y*)% 


ao —2y/(x — y)*; Z = 2x/(x — y)? 


D,f (x) =a,, where a =(aq,..., Qn) 


Dif(%) = 2 > ag;x; 
j=1 


of 


2x Of 2y 


xe +f y?? dy x+y 


2X 


— = ae (x?) ; a ~ 5003 (x?) 


2 2 2 
2x x of xe x 


= —;3; = =——sec* — 


Ox 


Sec 
yy’ oy youd 


+) 
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of y_ . Ff__ 3 
14. ~-=- > a = 
Ox xP ty? dy P+ y 
a Of 1+y? Of 1+? 
Ox +e ey tay? Wy Tt ty + xy 
of of 
= pty lt. 4 
16. ay Wade os ay 2yx" log x 
of 1 of Vx 
ee ee 
Ox 2Vx(y — x) dy 2yVy — x 
18. n= —-3 
19. a=b=1 


22. (b) One example is f(x) = x+y, where y is a fixed nonzero vector 


8.14 Exercises (page 262) 


1. (a) (2x + y®cos xy)i + (2y sin xy + xy? cos xy)j 
(b) e*cos yi — e* sin yj 
(c) 2xy8z4i + 3x2y?2z4j + 4x? y8z8k 
(d) 2xi — 2yj + 4zk 


2X ; 4y : 6z 


(e) x2 + 2y? — 322' we 2y? oe 3227 x2 + 2y? ee 332 F 


(f) y?x¥li + zy*1x"" log xj + y*x"" log x log yk 
2. (a) —2/V6 
(b) 1/V6 
(1, 0), in the direction of i; (—1, 0), in the direction of —i 
2i+2j; 414 
(a, b,c) = (6, 24, —8) or (—6, —24, 8) 
The set of points (x, y) on the line 5x — 3y = 6; Vf(a) = Si — 3 
(c) Yes 
(d) f(x, y,z) =$(x? + y? + 2”) 
11. (b) implies (a) and (c); (d) implies (a), (b), and (c); (f) implies (a) 


COON OY 


8.17 Exercises (page 268) 


pe of 


af as 
oy 


o°f 0? 2 ” 
SOMOS G8 3 [Y(t] tae Ota 


Ox Oy Oy 
2. (a) F’(t)=4t3 +2t; F'(t) = 1207 +2 

(b) F'(t) = (2cos?t — 1)e%S*8in! cos(cos t sin?t) + (3 sint — 2 sin t)e°S #8! sin(cos ¢ sin*f) ; 

F"(t) = (Scos*t — 3 cos*t — 4cos?t — cos? t — 4cos re 8! cos (cos t sin* ft) 

+ (14sin? ¢ — 12 sin t — 4sin t + 7 cost — 9 cos? t)eS!8in! sin (cos ¢ sin? t) 
2e*" exp (e%')  2e7* exp (e-*) 
1 + exp (e) 1 + exp (e7*) ° 
4[1 + e* + exp (e?")]e”* exp (ce?!) 4[1 +e ** + exp (e *)Je~* exp (e-*4) 
[1 + exp (e*’)}? [1 + exp (e-*4)} 


l(b) F°'(t) = a5 1X OP + 25-5 


() F(t)= 


where exp (u) =e"; 


F’(t) as 
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(a) —3 
(b) x? — y? 
(c) 0 
(a) (1 + 3x? + 3y?)(xi + yf) — (x? + y*)*k, or any scalar multiple thereof 
(b) cos@ = —[1 + (1 + 3(x? + y’))?}-'4; cos@—> iD as (x, y, z)—> (0, 0, 0) 
U(x, y) =4 log (x? + y”);_ V(x, y) = arctan (y/x) 
(b) No 
x/Xo + ylyo + 2/29 = 3 
x+y+2z=4, x-y-z=-l 
c= 4V3 
Exercises (page 275) 
(b) ay —2x sin (x? + y?) cos [cos (x? + y*)]Jesin [cos (2? + ¥*)] 
OF lof lof oF i10f 19of 
dx 2du 200° dy 20u 2a 
OF oOfox ofoY OF of ox of oY 
@) 3, 3x05 ‘dy os? Ot Ox ort dy OF 
OF Of 0X 0X Of (OX0Y OXdY Of oY oY Of Xx Of oFY 
tc Se er Ce ee Oo Ot a) a ds Ot | Ox Ost’ dy ds ot 
aor of Of OF OF OF) OY OT. 
Ci ar a Oy’ Of an dy’ Os? Ox? Ox oy a 
OF df O*f Of OF df Of Of of 
Be — Oxt tax ay + Bt? Boor ae TM TOR ay tHe By 
‘ OF of lof OF Of sof, OF_ of ptt gees 
oa oe aay? Of On oy 0 °° Ot Onde Op? 
02F ras 238 Of sof 2s Of OF _. Oe sof of 1of 
axe ax dy THT! Bay? Bear Axt Dye Ax — Dy 
OF lof 10f OF 16f lof OF _ lef 10°F, 
O 3 -28n 42a? ~~ 2a +2? Bor 4921 432° 
OF 10°F 1 ef 10°f- Or lof 1 ef 10% 
O° 40x? 1 2dxdy | 4 dy?’ or? 40x? 2dxdy 4 dy 
O2 2 
oF cost 022 + c0s 6 sin os ay t se) + int 0 
20 2 2 2 2 
aa =r eos 0 sin 054 + rcos* oF £ —rsint 0x5 + ros Osin 954 
sino 4 008 05; 


10. 


11. 


13. 


14, 


15. 


OF 
or 


OF 
Ot 
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_ Of Ox rac oY Lo OZ OF of ox , fer of OZ 
~ Ox Or Y By Or Oz O° Os Ox Os By Os * Oz 0s” 


_ eax aay  aaz 
Ot Ox Of dy Of | Oz OF 
oY Fe _Y YY _Y YY 


(a) 


(b) 


OF 


Os 
(a) 


(b) 


OF 
or 


(a) 


(a) 
(b) 
P’ 


(a) 
(b) 


(c) 


(a) 


ap Ox Op ae" Os Ox dy * Oz’ or Ox” Oy Oz 


OF f Of oOf\ OF |. (of of Of\ OF Of of of 
or (Z eed): a 7 (3 ~ 35 - Be) | or = a(t - 3 +5 


af ax afay oraz ar _afax , a aY | faz 


~ ax Os * Oy Os * Oz ds’ at “ax Or | Oy OF Oz OF 


OF _ of of of OF _ of of of 
Be = 595 ag aes an _ + 2s" 
Cn a i: 


Os Ox ¥ By T az’ Of ox ay +5 < 
ofox oOfoY OF OfaxX oOfoY OF Ofd0x ofdoY 


“Ox Or tay or’ Os Ox Os | Oy Os? Ot Ox Or dy O 


OF Of OF Of OF Of 
dr ox’ Os Ox’ OF ~ Oy 


OF of >of OF of of OF of 5, OF 


Or ox dy’ ds ox Oy’ dr ox Oy 
eee. 


dr sox’ Oss Ox toy? Ort? Oy 


f(x, y,z) = xi + yj + zk, plus any constant vector 


f(x, y, Z) = P(x)i + O(y)j + R(z)k, where P, Q, R are any three functions satisfying 
=p,Q =4q,R =r 
entey Qett2u 1 4v 9w? 
Df(x, y) = ; Dg(u,v,w) = 
? 2cos(y + 2x) cos (y + 2x) —2u 2 0O 


h(u, v, w) = et tee? +3us+40—2u?t 4b sin (2v — u2 + 2u + 40? + 6w)j 


—3 0 9 
Dh(1, —1,1) = 
0 -—6cos9 18cos9 


v2w2  2uvw2 = 2uv*w 
2x 1 1 ; 
Df(x, y, Z) = ; ; Dg(u,v,w) =| 0 w*cosv 2wsinv 


1 22 
2ue” uze” 0 


(b) A(u,v, w) = (vv4wt + w@ sin v + u2e”)i + (2uv?w? + w? sin v + ute?”)j 


(c) 


Qu wt ‘ 


Dh(u, 0, w) = 
44> w2+2' 0 
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8.24 Miscellaneous exercises (page 281) 


1. One example: f(x, y) = 3x when x = y, f(x, y) = 0 otherwise 

2. D,f(0,0)=0; Df(0,0) = —-1; D,,f(0,0) =0; D,>.f(0, 0) does not exist 

3. (a) Ifa = (a,, a), then f’(O; a) = ad/a? if a, #0, and f’(O; a) = Oif a, = 0 
(b) Not continuous at the origin 


ox oy 
ek 
5. F(t) = a LX’(P + 3 a a [LX’(nP Y(t) +3 oe spa XK OW'OF 
08 / o°f / 77) 0? w" V4 V4 ” 
4 ay (OP + 355 X’OX'| + ins IX OY Ga) +X OY] 
+3 = YOVO+s of ~ X(t) + o y(t), 


assuming the mixed partial derivatives are independent of the order of differentiation 
6. 8 


dg of Of oa _Ff | of 02g of ; 0? 


tes Ou Ox. + By“ > Ov Ox. ~ ay” > ina Gat © ee er 
92 
- w5h +90 (bt) a=4, b=-43 


10. (a) 9 (t) =A MIP FLAC, yldy + BOA flr, BOD] dx 
(b) ’(t) = 2te?’(2e” — et — e) 
13. A sphere with center at the origin and radius Pe 2 
14. f(x) =x? 
Chapter 9 
9.3 Exercises (page 286) 


1. f(x, y) = sin (x — 3y) 
2. f(x, y) =ertu/2 — |] 
3. (a) u(x, y) = x*y’em™ 


(b) v(x, y) =2 + tog |*| 
5S. A=B=C=1, D=—3; f(x,y) = 93x + y) + (x — y) 
6. G(x, y)=x—-y 
9.8 Exercises (page 302) 


OX/dv = (1 + xu)/(x — y); OY/du = (1 — yo)/(x — y); OY/ov = (1 + yu)/(y — x) 
2. dxX/dy = —(1 +xu)/(l +u); OVidu = (1 — y)/(1 + yu); OVidy = (1 —x)/ +) 
0X O(F,G)/O(F,G) OY _ OF, G) /O(F, G)- OY O(F,G) /O(F, G) 

ov. ~ A(y, v)/ OC, y) On ~ O(u, x) O(x, y) ’ Ov O(v, x) O(x, y) 


been, 


4. T= +e ow — Ai) + 3/7k) 


9.13 


Oo a. Sy i 


21. 


22. 
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2i+j + a 3k, or any nonzero scalar multiple thereof 

Ox/du =0, Ox/dv = n/12 

n=2 

Offox = -1/(2y +2z +1); dffdy = -—2%y + 2)/(2y +2z +1); 

02f/(Ox Oy) = 2/(2y + 2z + 1)8 

0°z/(Ox Oy) = [sin (x + y) cos? (y + z) + sin (y + z) cos? (x + y)]/cos* (y + z) 
of DF +2xD,F Of DF + 2yD,.F 


Ox D,F+2zD.F’ Oy DF + 2zDoF 


DF=f'ix+geQ)); DF=f'Ix +e’); Dik =f'ix +e(y; 
D, .F =f"[x +e(lg'(); DeeF =f" lx +eQMlg’FP + fie + gg") 


Exercises (page 313) 


Absolute minimum at (0, 1) 

Saddle point at (0, 1) 

Saddle point at (0, 0) 

Absolute minimum at each point of the line y = x + 1 

Saddle point at (1, 1) 

Absolute minimum at (1, 0) 

Saddle point at (0, 0) 

Saddle points at (0, 6) and at (x, 0), all x; relative minima at (0, y), 0 < y <6; relative 
maxima at (2, 3) and at (0, y) for y < Oand y > 6 

Saddle point at (0, 0); relative minimum at (1, 1) 

Saddle points at (nw + 7/2, 0), where n is any nteger 

Absolute minimum at (0, 0); saddle point at (—4, —4) 

Absolute minimum at (—#s, —;';); absolute maximum at (1, 3) 

Absolute maximum at (7/3, 7/3); absolute minimum at (27/3, 27/3); relative maximum at 
(7, 7); relative minimum at (0,0); saddle points at (0, 7) and (7, 0) 

Saddle point at (1, 1) 

Absolute maximum at each point of the circle x? + y? = 1; absolute minimum at (0, 0) 
(c) Relative eae at (2,2); no reas minima; Séddle Roe at (0, 3), (3, 0), and (3, 3) 
Relative maximum } at (4, 4) and (—4, —4); relative minimum —} at (4, —4) and (—43, 3); 
saddle points.at (0, 0), (£1, 0), and (0, +1); absolute maximum { at (1, —1) and (—1, 1); 
absolute minimum —1 at (1, 1) and (-—1, —1) 

(a) a=1, b=-% 

(b) a=6log2 —37/2, b= 7 —3log2 


n 


Let xt =1 > x, yt =n u, =x, —x*. Then a = (9 IC it), 


i=l i=] i=l] i=] 


1 I~ IX 
Let x* =- > x, yea“) y, a) x, u; =x, —x*, v,; =y;, —y™, and let 


2 
Se Soe 
2 
San De 


, where the sums are for i = 1,2,...,m”. Then 
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25. 


9.15 


1. 
Zi 


OS a ae ae 


Answers to exercises 
' due, > Uv; \ ~ v,2Z; > Uv; 
i= , b=- : = z* — ax* — by* 
“]Dee Det] Due Dd 
v,Z; vs M,Z; us 
Figenvalues 4, 16, 16; relative minimum at (1, 1, 1) 


Exercises (page 318) 


Maximum value is +; no minimum 
Maximum is 2; minimum is 1 


2 2 2 2 
(a) Maximum is —— at (b(a2 + b)-%, a(a? + b2)-%): minimum is — — at 
a a. 
(—b(a2 + b2)-%, —a(a? + b2)-”) 
ab? a*b 


ome : 242 2 2 ——<—<—— SS 
(b) Minimum is a*b*/(a? + b*) at (oa ga 


; no maximum 

Maximum is | + . 2/2 at the points (nw + 7/8, nz — 7/8), where nis any integer; minimum 
is 1 — ./2/2 at (nw + 5x/8, na + 37/8), where nis any integer 

Maximum is 3 at (4, —%, 3); minimum is —3 at (—4, 2, —#) 

(0,0, 1) and (0, 0, —1) 

1 

(i, 0, 0), (0, I, 0), (-1, 0, 0), (0, —1, 0) 


a*bh®¢° a b C 
atb+orr lot pte’ atbte’ atbte 
abc,/3/2 
5logr + 3 log /3 
wea +C—./(A —C)? + 4B? 

2(AC — B?) 


(4 + ./5)/./2 


Angle is 7/3; width across the bottom is c/3; maximum area is c?/ (4,/ 3) 


Chapter 10 
10.5 Exercises (page 328) 
—13 8. 3 
—2 na" 9, — 382 
$s 10. —27 
4 11. O 
0 12. (a) —2/27 
40 (b) —7 
23 


ey ae 


i] 
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10.9 Exercises (page 331) 


l 
2; 
3. 
A 
5 


16. 


-o 8. 256a3/15 
2a® 9, 2%a3(1 + 272) 
a = (3c/2)” 10. [(2 + 42)% — 2,/2]/3 
0 12. moment of inertia = 4a4 
87(sin 6 — cos 6) 13. 27/3 
5/4 14, 000. = 36./2 — 49 log (9 — 4,/2) 
7Q CO ——— 
64[6./2 + log (3 + 2,/2)] 
wp ee ee 6ab? a eas 6 zrab? 
~ 7 3a + 4ntp2? 7 


~~ 32 + 47%h2 


I, = (a2 + b?)“[na* + (423 — 2/2)a*b? + 327°b*/5] 
L, = (a + b?)4 [rat + (42° + 2/2)a2b? + 327°b4/5] 


10.13 Exercises (page 336) 


All except (f) are connected 

(a) Not conservative 

(b) (2e27 — 5e™ — 5x — 3)/10 

(b) 3 

g 

4b? — 87b + 4; minimum occurs when b = 7 


10.18 Exercises (page 345) 


—_—= —_— pf 
a es 


— 
we 


15. 


16. 


Po Sy es 


oxy =3P+y)+C 
(x,y) =x*y tC 


g(x,y) = xXe7 +xy — yr tC 

g(x,y) =xsiny + ycosx + (x? + y*)/2+C 
p(x, y) = xsin (xy) + C 

Ox, y, z= HPF+y4+2)2+C 

p(x, y,z) = x7/2 — y*/2 +xz —yz+C 

f is not a gradient 

f is not a gradient 

f is not a gradient 

p(x, y,z) =y*sinx +xz° —4y +2z+C 
@(X, y,Z) =x + 2x%y — x82 4+ 2y —- 274+ C 


n+1 
(b) oy) = 5 + Cifn #13 ox, y) = alogr + Cifn = —1 
pt+2 
ae TE Rc ce ll p(x) =logr + Cifp = —2 


P(x) =e) + C 


10.20 Exercises (page 349) 


1. 
2 


7/2 + 2xy + /2=C 
xy =C 
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3. x3/3 — xy — y/2 + (sin 2y)/4 =C 
4. cos2xsin3y =C 
5. xby + 4x?y? — 12e” + 12yeX = C 
6. f O(x)elP\aaz dx — yelP\z)de =-C 
8. (a) x ty =Cy* 
(b) yi/x* — 3 log |x| = 
9. (a) 6(xy)4 — (y/x)* =C; (x5y)-% is an integrating factor 


(b) x +e*siny =C; e cosy is an integrating factor 
10. x8yf+ x49 =C, 10x%y4 + x*y? = C, respectively; x*y* is a common integrating factor 


Chapter 11 
11.9 Exercises (page 362) 
1. 3 7. 6 
2. 1 8. te” —et) +62 — 7 
3, 2/3 -— 3 10. 4 
4, a2/4 11. $3 — /2) 
5. 2 12, 7/2 
6. 27 13. (log 2)/6 


11.15 Exercises (page 371) 


1. —37/2 5. wv — 42 

2. 2+.cosl1 +sin1 —cos2 —2sin2 6. 6 

3. e—e} 7. 32 

4. glog2 8. (a) $ (b) 2 (c) 320m 


2 [[Rnene] a 

10. {, ee f(x, y) dy dx 

un. {* [ [" fee, y) dx dy 

12. i ings I(x, y) dx | dy 

o Me rena a+ [Fare nae] 4 
14. Rs I (x, y) dx | dy 

PLM re.na]ors Ere nar]o 
16. fo] [ee fee, y) dx | ay 

P i" 2 arcsin yl % y) dx | dy 5 if | en “fe, y) dx dy 


[Lee £2) 4y] a 


—_ 
= 


b—_ 
a 


Answers to exercises 649 


1 2—2 2 9 _4 
19. [*| [?* Gt + yay] dx = 4 
20. y=0, y=xtane, w*+y=a*, 2+y? =6? 


21. (a) inl Ries y) dx] dy 


(b) 4e® + 2e/3 
22, m=2; n=1 


11.18 Exercises (page 377) 


1 x=-4t, 7 =#% 
2 x=1, y=O0 
3. F=18, P= 3s 
4. X=7/2, 7 = 7/8 
2 2 : 241 
5. £=(/2 +1) al = oe —-1- 2, yest 
4 2 4 
bi ren 2a*loga—a®? +1 a(log a)? 
: * 4(aloga —a +1)’ ”**Qaloga—atl) 
7. F=f =3 
8. x = a = 256/(3157) 
9, 38 — *# log 3 
10. x =4% ABI, jy =% AD| ; assuming the x- and y-axes are chosen along sides AB and AD, 
fecal 
5m 27° 
11 Bh="p Fas 77 
12. I, =i2b(a—c), I, = 2b@ — c) 
13. LZ=L=d - 577/16)r4 
14, I, = is = 2 
15. 1, =¢al(4a — Ie —-— 1], 1, =ael(a@® — 3a? + 6a — 6)e** + 6] 
_ 72 _ 148 
16. I, =108, I, =a 
19. $h[/2 + log i + /2)] 
20. hh? + $r? 
21. (a) (é*,1) 
(b) G,3 
(c) Ce, *#) 
(d) (42 13 
22, kh =2./3 
23. h>r,/2 


11.22 Exercises (page 385) 


1. (a) —4 
(b) 4 
(c) 8 
(d) 47 
(e) 32/2 


ai ral had 


Answers to exercises 


— 7 


&(x, y) = +[P*(x, y) + Ox, y)]” 


% 11.25 Exercises (page 391) 


a ae 


(b) 0 

0, 27, —27 
As many as three 
As many as seven 
(a) -3 

2a 


11.28 Exercises (page 399) 


bd 
e 


we 


= 


_ 


10. 


11. 


12. 


13. 


14. 
15. 


17. 


2 SD) 


[° [ [240 cos 8, x sin 6)r dr | a6 


FTP? pcos 6, vsin Or dr | dO 


a/2 


(he ere cos 9, r sin @)r dr | dé 


ie Le f(rcos 6,rsin 6)rdr|d@, where g(0) = 1/(cos6 + sin 6) 


| os | { nan 28°°9 Fy cos 8, r sin 8)r dr] dO + ee os [. °° F(r-cos 8, r sin 6)r dr | dO 


0 0 


+ ae i aaa ic cos 6, r sin 0)r dr | d0 
ima‘ 
3a3[./2 + log (1 + ./2)] 
J2-1 
7a*/8 


‘ee ie f (r cos 0, rsin 4)r dr | dé + ie | : f(rcos 6, rsin 6)r dr | dé 
(oP porar] a6 


Me [ ee f(r cos 6, r sin 6)r dr | dé, where g(9) = 1/(cos 6 + sin 6) 


0 


hae iia °°? f(r cos 6, r sin 8)r dr | dd 


npeeael’ 


Answers to exercises 651 


18. (a) 4(u? + v?) 
(c) 0 


19. m7 [(p2 + 2)? — p-P] ifp #1; wlog( +P) ifp=1. 


I(p, r) tends to a finite limit when p > 1 


11.34 Exercises (page 413) 


WR WN 
2 


© Ue Lo fy-2ddy] de + J LTP 2b ae} a 
7» [APU acaresy. 2a] a a 


BLUE Lorem zndy] de + [2 fy mal sey. 2) ay] a 


10. 16/3 
ll. 4 
12. dswa*h(3a? + 2h?) 


6 
47a 

14. 4n(b* — a?) 

15. 47R3(a? + b? + c2)-% 

18. 3(5,/5 — 4) 

19, 43 

20. 41(b° = a°) 

22. On the axis at distance 2h from the base 
23. On the axis at distance +h from the base 


3 bt —a‘ 
24. On the axis of symmetry at distance 3 ; Boa from the “cutting plane” of the hemispheres 
25. x = jy = Z = 7'sh (assuming the specified corner is at the origin) 
26. 35M (a? + 4h) 
27. 2MR?® 
28. 2Ma? 
29. 2% 
Chapter 12 


12.4 Exercises (page 424) 
1. (aobz, — Asby)(x = Xo) + (a3b, os aybs)(y Yo) + (a,b, sd Ayb,)(z = Zo) = 0; 


or Or 
= av = (agb5 7x asb.)i + (ab, oars a,bs)j + (a,b, ae ab, )k 


or 
2. x®/a® + y?/b® = z; Poa —2bu? cos vi — 2au? sin vj + abuk 
v 
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Answers to exercises 


a a i or or he sinucosv,  sinusinv , COS 
mptpta > 5, 5p 7 abe sin u i+ b J+ 
fies ee t= uf"(u) sin of + uk 
z=f(/x* + y*); = 5, = Uf ) cos vf — uf (u sin oj + u 
2: ane 4 r or ae 
mt poh oy * 7p 2 Sin tacos vk 
(/x? + y? —a? + 22 =3B; 
or ee ae 
5, % 3p 7 DUG + bcos u)(cos u sin vi + cos u cos uf + sin uk) 
sin?u cos? u ; sinh? v |] 
labc| cosh v = ta cosh* v + 2 
/128v? + 4 
lu — v| \/36u2v? + 9(u + 0)? +4 
Jub +2 
Exercises (page 429) 
na? /'3 
(27 — 4)a? 
4 


(a) Accircular paraboloid 

(b) —2u? cos vi — 2u? sin uf + uk 

(c) n=6 

J/2 na?/4 

27/6 

2na2(3./3 — 1)/3 

4n*ab 

(a) A unit circle in the xy-plane; a unit semicircle in the xz-plane, with z <0; a unit 
semicircle in the plane x = y with z < 0 

(b) The hemisphere x? + y? +z? =1,z <0 

(c) The sphere x? + y? + z? = 1 except for the North Pole; the line joining the North 
Pole and (x, y, z) intersects the xy-plane at (u, v, 0) 


12.10 Exercises (page 436) 


2 ee a 


On the axis of the cone, at a distance ta(1 — cos «)/[1 — cos («/2)] from the center of the 
sphere 

na°h + gzah? 12. 27/3 

3na®h + $rah' 13. —7/3 
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12.13 Exercises (page 442) 


: 3. 
2. —-7 4. 


©2| 


12.15 Exercises (page 447) 


1. (a) div F(x, y,z) =2x + 2y +2z; curl F(x, y,z) =0 
(b) div F(x, y,z) =0; curl F(x, y, z) = 2i + 4j + 6k 
(c) div F(x, y,z) = —xsiny; curl F(x, y,z) =i+j 
(d) div F(x, y, z) = ye” — x sin (xy) — 2xz sin (xz?) ; 
curl F(x, y, z) = z* sin (xz*)j — [xe™ + y sin (xy)]k 
(e) div F(x, y,z) = 2x sin y + 2y sin (xz) — xy sin z cos (cos 2) ; 
curl F(x, y, z) = [x sin (cos z) — xy’ cos (xz)]i — y sin (cos z)j + [yz cos (xz) — 
x? cos y]k 
2. 0 
4, = —3 
5. No such vector field 
10. One such field is u(x, y, z) = (xyz)? 
ll. div(Vxr)=0; curll(V xr) =(c+1)V 
13. 16(a + bd) 


*%12.17 Exercises (page 452) 


1. (3x — 2z)j — xk is one such field 
.  (x7/2 — xy — yz + 27/2)j + (x?/2 — xz)k is one such field 
3. (x®y/2 + z?/2)j + VF(x, y) for some f independent of z 
5. G(x, y,z) = a? _ EES) j satisfies curl G = r~*r at all points not on the z-axis 
6. f(r) = Cr? 
9. F(x, y,z) = —3(278 + xf + y’k), Gx, y,z) = $V y + y®z + 23x) 
10. (c) 3z/2 


12.21 Exercises (page 462) 


i; .3 3. (a) 31V| 
2. (a) 1447 (b) 9|VIZ 
(b) —167 (c) |VIx 
(c) 1287 (d) 4/, 
15. 87 
Chapter 13 


13.4 Exercises (page 472) 


n n—1 n 
2, A, UA, U Ag = (Ay 1 AZO AQ) U (A, 0 AZ) U AZ3) U A, = U (4, 1) AD UA, 
k=1 k=1 j=k+1 


3. (i) (ii) (iii) (iv) (v) 


(A 0 B’) VU (A! OB) 


300 800 


(a) A’. OB’ 


(b) 500 
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B 
B 


Answers to exercises 


13.7 Exercises (page 477) 


— et et pe 
PMS See a See ee 


Zi 


10. 
12, 


AC B’ 
xEA OB OC 
xEANB OC’ 
XEAUBUC 


PEt A ZA nA UA, A YA A AS) 
3 ={@,A,,A,, As, Ay U A,, Ay U As, Ay U Ag, A, U A, U Ag, Aj, Ay, As, Ay OAs, 
50. A3, A, O45, AL OAS OV AG, S} (if. 2 > 3) 


XE(ANBODC)UAABOC)U(A AB OAC) 


xE(AOB)VU(BONC)VU(A OC) 
xXxE(ANB)YV(ANC)YU(BOC) 


XE(ANBONC')VUANBOANOQVU(A ABAC) 


xE(ANBOC) 
xEANCHB 


xEANBNOC 
xXxEAUBUC 
(a) l—a (d) l—e 
(b) 1—b (ec) l-—ate 
(c) atb—-e (f) a-—e 
13.9 Exercises (page 479) 
(a) 5. a7 
(b) $¢ 6. 44 
(c) $2 1. ff 
(d) $¢ 8. (a) A/(A + B) 
(a) 3% (b) B/(A + B) 
(b) ¢% (c) (C+ 1)/(C + D +1) 
(c) 4 (d) C/(C+D +1) 
(d) 3% 9. (a) ¢ 
(e) 38 (b) ¢ 
(f) 2 (c) ¢ 
Z (d) 3 
P) =1—P(A) — P(B) + P(A OB), P, =P(A) + P(B) — 2P(A OB), 
(a) Sto9 
(b) 45 to 46 
(c) 10 to 81 
(d) 36to 55 


13.11 Exercises (page 485) 


DS 


{(1, 2), (1, 3), (2, 1), (2, 3), 3, D, G, 2)} 
1326 
54 


{H, T} x {H, T} x {1, 2, 3, 4, 5, 6}; 24 outcomes 


521/(13!)4 


P, = P(A OB) 
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7. (a) 13-+12-11 +72 = 123552 (not including triplets or quadruplets) 


(b) 5148 
(c) 36 (not including 10JQKA) 
(d) 4 
8 P 13 52 b) 36 52 4 52 
© 4('s)/(s) © 36/(5) © 4/(5) 
2-98! 98! 
9. (a) ——__— 


aope ©) aer-sor 


10 98 100 
" \48 50 

11. 16 

12. nk 


13.14 Exercises (page 490) 
2. (a) P(A) =3;; PIB) A=; P(ANBD=% 


(i) 


ss 100 98 
50) ~ (so) 
5. it 
34 62 
26! - 34! (13) (‘ 
Se MDP pip eggy = fad = 780) 


(5) 


$ 
15. (a) P(A) = P(B) = P(C) =3; P(A NB) =P(A NC) =P(BNC) =}; 
P(ANBONC)=0 


13.18 Exercises (page 499) 


1. (a) PCH, H)=pip2; PCH, T) =p, — pe); P(T, H) = (1 — pips; 
P(T, T) = (1 — p,)(1 — ps) 
(b) Yes 
(c) No 
(d) H,and H,, H, and T,, Hz and 7T,, T, and T, 
2. (a) #12 
(b) yoda 
(c) 6 


656 Answers to exercises 


10\ 57 390625 n\ wkpn-k 
3] 69 ~~ 2519424 k}(w +b)" 

4. (a) 3% 8\ 175 9938999 
(b) 3 3) 188 — 1377495072 
(c) : 

5. (a) (512/10! = 545 10. $23 
(b) 4 11. 1 — (19/20)° = 0.4013 

6. (a) 36p!° — 80p® + 45p® 12, 193 
(b) shy ° 512 


7. It is advantageous to bet even money 14. 59 <n < 65 


15. (a) f(p) = (1 — p)*? + p® 
(b) (V31 — 4)/3 


13.20 Exercises (page 504) 


1. (a) f(k) = 2k 
(b) f(k) = 3* 
(c) f(k) = px, where p, is the kth prime >2 
(d) one such function is f(k) = (g(k), h(k)), where 
m(k) + 3m(k) : _ m*(k) + m(k) 


5 k+2, h(k) =k 5 


8k —-7—-1 
ne = [EN 


where [x] denotes the greatest integer <x 
(e) f(k) = 29'*)3") | where g(k) and h(k) are as defined in part (d) 


&(k) = 


9 


and 


13.22 Exercises (page 507) 


ln=0: max=1, min=}3 
n=1: max=}, min =0 
=2: max=3, min=0 
n=3: max=7g, min =0 
3. (a) 1 —gp® — pq 4. (a) 3pqi(pq + 2) 
(b) % (b) 3 
(c) % (c) 2log2—1 
13.23 Miscellaneous exercises on probability (page 507) 
a 8. (a) 3 
(a) i (b) 4 
(b) zo (c) 3 
3. (a) } (d) No 
(b) i% l—p 
4. (a) 33 9. pi + (> ) 
(b) $7 sy = 
(cy) 2% 10. np(1 — p)”* + mp1 — p) 


5. $4 n n \r 
7. 0.65 ee pa (1 - 55) 


Answers to exercises 657 


Chapter 14 


14.4 Exercises (page 513) 


1. 


_ 


(b) X <b 

(a) {w| X(w)€ (a, db], Y(w) € (c, d]}} 

(c) X<a, Y<d 

(d) Pla<X<b,c< Y<d)=P(X <b, Y <d)—P(X <a, Y <d)—P(X<b, Y <0) 
+ P(X <a, Y <0) 

(a) {(1, 6), (2, 5), (3, 4), (4, 3), G, 2), (6, D}, {6, 6), (6, 5}, {, 6), (2, 5), (3, 4), , 3), 
(S, 2), (6, 1), (5, 6), (6, 5)} 

(bt) P(X =7) =%; P(X =11 =7; P(X = 7 or X = 11) =3 
Y=X,+X,+X,+%; PY=0 =r; POY =VD=4; PY <)D=% 

Y=7X if0 <X <100; Y=10X — 300 if X¥ > 100 

(a) Z=Y-1 

(bt) U=Y,+Y,-1 


14.8 Exercises (page 523) 


23 


10. 


11. 


t 2 3 4 5 6 7 8 9 10 11 12 
PAt) | se te re 3s 38 6 38 & TS I 
(b) p(—2) =4, p(O) =p(2) =} 

(c) 0, 3, 3, }, 1, 0, +, 4 

(a) c=3 

(b) $,3,3,8 

(c) F(t)=Ofort <0, F(t)=$for0 <t <1, F(t)=4forl <t <2, F(t) =1 for 
t>2 

(d) No such ¢ 

(ce) t=2 

(a) &k 0 1 2 3 4 


ptk) at ot Pi Fa a p(t) _ 0 for t ¥ 0, 1, 2, 3,4 


(b) F(t)=Ofort <0, F(t) =4£0n [0,1), F(t) =4%0n [l, 2), F(t) = % on [2, 3), 
F(t) = 3{ on B,4), F(t) =1 fort >4 

(c) 7; at 

(b) P(X =0) = (1 —p)*; P(X = 1) = 2p(1 — p) 

[¢]([¢] + 1) 


2k 
Px lk) = "n+ 1) ; Fy() = ae for 0 <t <n, where [t] denotes the greatest 


integer <¢t; Fx(t) =Ofort <0; Fxy(@t) =I1fort>n 


ke 
c 
Pk) =e" k =0,1,2,3,...; c>090 
p(t) = 0 for t 4 0,1, 2, 3,... 
(a) px(t)=4$at¢t = —landt = +1; px(t) = 0 elsewhere 
Fy(t) =Ofort < —-1; FS SOEs Fy(t)=1fort>1 
P(A) =§; P(B)=$; P(A NB)=3; P(B| A) =}3; P(AVB)=1 
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14.12 Exercises (page 532) 


1, 


Zi. 


3. 


10. 


(a) c=1; fgo=1ifO ctl; f(t) =0 otherwise 

(b) 0,3,3 

c=3; F(t)=O0ift < —nx/2; F(t) =}costif —n/2<t <0; F(t) =1—}4cost if 
O<st<-7f/2; F(t) =1lift > 7/2 

c=2%; F(t)=0 if ¢<0; Ft) =iBe—-AifO<t<2; F®=1ift>2 


(a) 3 

(b) 3 

(c) } 

(a) f(t)=Oift <4; f(t) =8 -—lifk <t<}; fi) =7 — 8tifk <t <3; f(t) =0 
ift >? 


(b) F(t) =Oift <4; Ft) =40?-tif- st <4; FC) = —4t? + 7t — 2if4 <t < 3}; 
F(it)=1ift >? 

(c) 1,1,4,0, is 

(a) 0,8, %,4 
C= 


(b) 1 
(a) k 5 10 15 20 25 #430 
PX>k| #@ 3 4 $ é 0 


(b) Let each Styx train arrive 5 minutes before a Lethe train 

Fy(t)=0ift<b; FyOM=C¢—bljaifb<t<cbt+a; Fyt=lift>b+a 
(a) 
(b) 
(c) 


a|eo col ins 
NO ale a 


t—b 


1 1 
F(t) = 5 + = arctan 


14.16 Exercises (page 540) 


1. 


2. 


2a > 


(a) 105 

(b) 10.05 

(a) $(/2 -1) 

(b) 3(2 — /2) 

(c) 32 —- Ki 2) 

(d) 24(4 — ./2) 

(a) 1—e? 

(b) e? 

(c) (e — 1)/e? 

(d) e&3 

Fit) =0ift<c; Fit) =1—-—e“%*oOift>c 
NV =ifa, c =b+ac 
(a) 0.5000 

(b) 0.1359 

(c) 0.9974 

(d) 0.0456 

(a) 0.675 

(b) 0.025 

(a) 0.6750 

(b) 0.025¢ 


12. 


13. 
14. 


15. 


Answers to exercises 


(a) 0.8185 

(b) 0.8400 

75.98 inches 

mean = b, variance = a” 

Fy(t)=O0ift <0; fy(t) =0ift <0; fot) =e*/,/2at if t > 0 


14.18 Exercises (page 542) 
(a2) Fy@)=0ift <1; Fyp@M=C¢-DPifl<1<4; Fyp@M=1ift>4; f@ =Fif 


1. 


1<t<4; fy@) = 0 otherwise 
(b) Fy@) =0ift < -2; FpM=(4+2)/3if —-2<t<1; Fyp@=lift>1; 
fyr@® = if —-2<t<1; fy@) =0 otherwise 
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(c) Fy(t)=Oift <0; Fy(t) =1%if0 <t¢ <1; FyQ)=1ift>1; fy@ = Qt“ if 


0<t<1; fy@ =0 otherwise 


(d) Fy(t)=eifte <0; Fy =1lift>0; fp =eifr <0; fp) =0ifr>0 


(e:) Fy) =e? ift <0; Fyp@=lift>0; fp@ =heift <0; fVF@ =O0ift 


> 0 


(ff) Fyp@=0ift <1; Fy) =logrifil<t<se; Fy) =1lift>e; fp@ =I1/tif 


l<t<e; fy(t) = 0 otherwise 
Let y be the inverse of ¢, defined on the open interval (a, b). 


fxlvOly’(@) ifa<t<b; fy(t) = 0 otherwise 

(a) fp) =Oift <0; fy) = (at) %e"” if t > 0 

(b) fet) =O0ift <0; fy) = 4tQzr)-“%e"” if ¢ > 0 

(c) fyp(t) =O0ift <0; f(t) = mt?) %e-M08 47/2 if ¢ > 0 

(d) fy (t) = (27)-% sec? t e(tan’ 0/2 if |r] < n/2; fy (t) = Oif It] > 7/2 


14.22 Exercises (page 548) 


Z: 


10. 


Oo NY 


(a) P(X = x) = P(X = x2) = P(Y = yy) = P(Y = yo) = 3(p + 9) 
(b) p=q=3 


Fa.) =| — || — Vitae eo and d 
(a) OW=|5_-,] qaocjuas*s and csy <4, 


x 
5b d—c 


F(x,y)=1lifx >b and y>d, F(x, y) = 0 otherwise 


ye 


F(x, y) = 


—a 
7ifasx <b and y >d, F(x,y) =—=—— _ifx > 5 andc <y <d, 


(b) Fxy(x%) = (x -a/(b-—aifacsx <b; F(x) =0ifx <a; Fy) =lifx>b; 
Fy) = -oO/d-oife<y<d; Fpl) =0ify<c; Fy(y=lify >d 


(c) X and Y are independent 
P(Y=0)=4; P(Y=1) =P(Y =2) =} 
1 4 


6° 6 


3 

(b) f(x,y) =2if & yNEQ; f(x, y) = Oif &, WEQ; 
fx) =1 — [x] if |x] <1; fx(%) = Oif |x| > 1; 
fr =1-lylifbyl <1; fyQ) = Oifly| >1. 
X and Y are not independent 

glu, v) = f(u +a,v +b) 


pms 


14.24 Exercises (page 553) 


Ly 


(b) fp&) =1+vif -l1<v <0; ff&@=1—-vifO<v <1; ff =Oif |v] >1 
(c) Uand V are not independent 
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2. (b) fp) =2 —2tif0 <t<1; f(t) =0 otherwise 
(c) Uand V are independent 
3. (b) glu,v) =ue*ifu>0, 0O<v <1; g(u,v) = 0 otherwise 
(cc) fy@ =ue"“ifu>0; fyW =0ifu <0 
(d) fp) =1if0 <v <1; fv) = 0 otherwise 
4. (b) gu, v) = (v/2m)e“OAt)e*/2 if y > 0 
(c) fy@ = [nl +P 


5. fg(t) = 17" get 
2 


6. (b) fy(u) =Oifu <0: Fos) = ulo®)exp ( - 55) ifu>0; Fy(t)=O0ift <0; 
2 
Fy(t) = 1 — exp (- 55) it 20 


14.27 Exercises (page 560) 
1 E(X)=4, Var (x) = 2% 


7. (a) E(X) = Var (X) =A 
(b) None 
(c) E(X)=1/A, Var (X) = 1/2? 
(d) E(X) =m, Var (X) = eo 
8. (a) C(r) =(r —1)/2 
(b) Fy(t) = lt ift << -1; Fy =tif -l<r<s1; Fy@)=1—-43¢ift>1 
(c) P(X <5) =1 —S" 4/2; P(S < X < 10) = (5'* — 10!-”)/2 
(d) X has a finite expectation whenr >2; E(X)=0 
(e) Variance is finiteforr >3; Var (X) = (r — 1)/(r — 3) 
9. E(X) = E(Y) = —=F; E(Z) = —1767/50653; E(X + ¥Y + Z) = —4505/50653 
10. E(X)—> oasn—> o 
12. (a) (2/7)% 
(b) e% 
(c) e*—e 
(d) (n/2)% 


14.31 Exercises (page 568) 


4. 251 

5. 0 

6. Chebyshev’s inequality gives 4; tables give 0.0027 

8. (b) 0.6826 

9. (b) 0.0796 

10. (a) 0.0090 
(b) 0.0179 

Chapter 15 

15.5 Exercises (page 577) 

2. (a) No 3. (a) Neither 
(b) Yes (b) Seminorm 
(c) Yes (c) Seminorm 
(d) No (d) Neither 
(e) Yes (ec) Seminorm 
(f) No (f) Norm 
(g) Yes (g) Neither 


(h) Neither 
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an 


(a), (b), (©) 

(b) The polynomial in (a) plus $(5x? — 3x) J1, (St? — 32) f(t) dt + ~3,(35x* — 30x? + 3) 
x Ji, (3524 — 30t? + 3) f(t) dt 

9. a= —3e/4 + 33/(4e), b=3/e, c =*B(e —7/e) 

10. @3(1 + 14x? — i 

1. (b) If — PIP = de 


n—1_ log?n 


a 


12. (a) ||P—-f\? = 


n—I1 
_ 12 6(n + 1) 4(n? — 1) logn 6(n + 1)_ 
” PO) = (Ga Gop 8) @—bht  ~ @—De 
||P — — es ee 


13. (a) ||P —f\l? = - [(n — 2)e2” + 4e" —n — 2] 
(b) P(x) = (18 — 6e)x +4e — 10; ||P — f ||? = 20e — Ze? — Sf = 0.0038 


J@k + Dk +9) » wert 
kel XOX) — FT rae ae Py-s(x), Where g = 


14. (b)  Pys(%) = 
Pf || Pell 


15. (b) dn = = J Pnla)Pnda(®) — Pra P i] 


15.9 Exercises (page 585) 


1. (a) P(x) = (x? + 13x + 12) 
(b) P(x) = 4(x? — 5x + 6) 
(c) P(x) = —Z(x3 — 6x? + 5x — 6) 
(d) P(x) =2x°+x%-—x-2 
(e) P(x) = —5x® — x7 + 10x —5 
2. P(x) = ge_(9x* — 196x? + 640) 
4. (a) Q(x) = 2x? + 3x7 —x —3 
(b) O(x) = 4x" + 7x? — 3x -—7 
5. (a) P(32) = 34; Rice P(32) = 43 


(b) P(32) = 2292; (32) — P(32) = —702 
(c) P32) = 48; G2) — P32) = 39, 
(d) P(32) = os 03; f(32) — P(32) = 2334 


7. (a) Lo) =a —- Deu -3u-4)u - - 6); L(x) = —zgu(u — 3)(u — 4)(u — 6); 
L(x) = pgutu — Iu —- 4) — 6); L(x) = —ygute — DU — 3)u — 6); 
L,(x) = aigu(u — 1) — 3)u - 4) 
(b) P(2.6) = 20 
8. (b) x > 1.581 
(c) h < 0.0006 
12, (a) a=0, b=1 
(b) c=1, d= —2L,(x;) 
13. (d) Let By(x) =1 and let B,(x) = (x — x(x — xX) — nh)” Jn! for n > 1; the one and 
only polynomial P of degree <n satisfying the conditions P(x») = cy, P’(x;)) =a, 
P"(Xq) = €g,..., P'™ (Xn) = Cy is given by P(x) = coBy(x) +--+ + CnBn(x) 
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Answers to exercises 


15.13 Exercises (page 593) 


4. (b) 


(c) 


(d) 
7. (a) 
(b) 
(c) 
(d) 


8 —5040 13068} —13132 6769; —1960 


322} —28} 1 


9 40320 | — 109584 118124} —67284; 22449|-—4536| 546;-—36) 1 


10 | —3628800/ 1026576) —1172700| 723680] —269325) 63273} —9450| 870)/—45| 1 


1 +2x + 2x? — 3x3 + x! 


8 | 1 127 966 1701 1050 266 


10 | 1 511 9330 34105 | 42525 | 22827 


—1 + 6x +4 16x) 4 9x3) 4 x) 
Sn3 + tn? + Stn 

int + 2n3 + 3n? + Sn 

int + 3n3 + +n? + 3n 

sn + 1nt + 1n3 — Jon 


15.18 Exercises (page 600) 


2. (b) 


5. sin @sinnd = 


T,, (1) = n*, T/(-1) = (—1)"-1n? 


1 — x? 


: T,(x); degree =n +1 


7. Q(x) = x" — 2-™Ty (x) 
8. Ox) = hat + Bet Bat + a — aly 
U(x) =1, U(x) =2x, U(x) = 4x7 —1, U(x) = 8x3 - 4x, 


14. (b) 


U,(x) = 16x* — 12x77 +1, U,(x) = 32x° — 32x3 + 6x 


15.21 Exercises (page 610) 


1. (a) 
(b) 
2. (a) 
(b) 
3. (a) 
(b) 


0.693773 — «, where 0.000208 < « < 0.001667. This gives the inequalities 
0.6921 < log 2 < 0.6936 
n = 578 
c= ./3/3 
a+b b-a,J/3 a+b b-aQJ/3 
a oY oy oe ee oe 
c= 22 
a+b b—a,/2 a+b b-—a,/2 
Cy = + os Cy eS ee 
2 Zz 2 2 2 2 


@=24.)2, BS2 = /2 


4. + 
5. c= /2 


10. 


11. 


(d) 


(a) 
(b) 
(d) 
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” Fx) d eee ls b+a b-a 3 3 b+a 
IOS ag No 2 V5) + ¥\-9 
15 a 3 
if ps 2 V5 
log 2 = 0.693254 — «, where 0.000016 < « < 0.000521 ; this leads to the inequalities 


0.69273 < log2 < 0.69324 

log 2 = 0.69315023 — «, where 0.00000041 < « < 0.00001334; this leads to the in- 
equalities 0.693136 < log 2 < 0.693149 
log 2 = 0.693750 — e, where 0.000115 
0.69004 < log 2 < 0.69364 


< « < 0.003704 ; this leads to the inequalities 


INDEX 


ABEL, NIELS HENRIK, 162 


Abel’s formula for Wronskian determinants, 


162 
Absolute maximum, 304 
Absolute minimum, 304 
ADAMS, JOHN COUCH, 571 
Adjoint of a matrix, 122 
Angles in a real Euclidean space, 18 
Annihilators, 151 
method of, 163 
table of, 166 
Approximate integration: 
by Cotes’ rule, 612 (Exercise 11) 
by Euler’s summation formula, 613, 
615 
by Simpson’s rule, 608, 609 
by the trapezoidal rule, 604 
Approximations: 
by interpolation polynomials, 579 
by Legendre polynomials, 29 
by Taylor polynomials, 576 
by trigonometric polynomials, 29 
in a Euclidean space, 28 
successive, 223 
ARCHIMEDES, 571 
Area: 
as a line integral, 383 
of a plane region, 368 
of a surface, 424 
Area cosine principle, 426 
At random, 478 
Attraction, gravitational, 335 
Average density, 374 
Average of a function, 374 
Average rate of change, 253 
Axioms: 
for a determinant function, 73 
for a linear space, 3 
for an inner product, 14 
for probability, 474, 506, 511 


Ball in n-space, 244 
Basis, 12 
Bayes’ formulas, 549 (Exercise 6) 
BERNOULLI, DANIEL, 182 
BERNOULLI, JAKOB, 469, 495 
Bernoulli trials, 495 
BESSEL, FRIEDRICH WILHELM, 182 
Bessel differential equation, 182 
Bessel functions: 

graphs of, 186 

of first kind, 186 

of second kind, 188 
Binomial coefficients, 481 
Binomial distribution, 521 

approximation by normal, 538 
Boolean algebra of sets, 471 
BOREL, EMILE, 510 
Borel set, 510 
Boundary of a set, 245 
Boundary point, 245 
Bounded function, 357 
Boundedness of continuous functions, 319 
Bridge hands, 481 


CAQUE, J., 223 
Cartesian product, 244, 
CAUCHY, AUGUSTIN-LOUIS, 16, 142, 191, 530 
Cauchy distribution, 530 
Cauchy functional equation, 528 
Cauchy-Schwarz inequality, 16 
CAYLEY, ARTHUR, 202 
Cayley-Hamilton theorem, 203 
Center of mass, 373, 374, 431 
Central limit property, 567 
Central limit theorem, 566 
Centroid, 374 
Chain rule for derivatives: 

of matrix functions, 194 

of scalar fields, 264, 273 

of vector fields, 272 
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666 


Change of variable: 

in a double integral, 394 

in a line integral, 327 

in an n-fold integral, 408 

in a surface integral, 433 
Characteristic equation, 150 
Characteristic polynomial, 103, 149 
CHEBYSHEV, PAFNUTI LIWOWICH, 563, 596, 611 
Chebyshev inequality, 563 
Chebyshev integration formulas, 611 
Chebyshev polynomials, 596 
Christoffel numbers, 610 (Exercise 6) 
Circulation, 330, 462 
Clockwise (negative) direction, 389 
Closed curve, 334, 379 
Closed set, 246 
Closure axioms, 3 
Coefficient matrix, 58 
Cofactor, 87 
Cofactor matrix, 92 
Coin-tossing games, 506, 558 
Column matrix (column vector), 51 
Combinatorial analysis, 481 
Complement of a set, 246 
Complex Euclidean space, 15 
Complex linear space, 4 
Components, 13 
Composite functions, 37, 194, 250 

continuity of, 250 

differentiation of, 194, 264, 272 
Composition of transformations, 37 
Compound experiments, 492 
Conditional probability, 486 
Connected set, 332 
Conservative field, 329 
Constant-coefficient derivative operators, 148 
Constant-coefficient difference operators, 595 

(Exercise 8) 

Content zero, 364, 406 
Continuity of scalar and vector fields, 247 
Continuous distribution, 525 
Continuous random variable, 525 
Continuously differentiable scalar field, 261 
Contraction constant, 235 
Contraction operator, 234 
Convex combination, 377 
Convex set, 344 
Coordinates: 

cylindrical, 409 

polar, 274, 397 

spherical, 293, 410, 414 
COTES, ROGER, 612 (Exercise 11) 
Cotes’ rule, 612 (Exercise 11) 
Countable set, 502 
Countably additive set function, 506 
Countably infinite set, 502 


Index 


Counterclockwise (positive) direction, 389 
CRAMER, GABRIEL, 93 
Cramer’s rule, 93 
CRAMER, HARALD, 520 
Curl of a vector field, 441 
Curve: 

closed, 334, 379 

in n-space, 324 

Jordan, 379 

piecewise smooth, 324 
Cylindrical coordinates, 409 


D’ALEMBERT, JEAN, 288 
Del operator V, 259 
DE MOIVRE, ABRAHAM, 469, 567 
Density function, 525 
for Cauchy distributions, 531 
for exponential! distributions, 533 
for normal distributions, 535 
for uniform distributions, 527 
Dependent sets, 9 
Derivative: 
and continuity, 260 
directional, 254 
of a matrix function, 193 
of a scalar field with respect to a vector, 253 
of a vector field with respect to a vector, 270 
partial, 254 
total, 258, 270 
Determinants, 71 
axiomatic definition of, 73 
differentiation of, 80 (Exercise 6) 
expansion formulas for, 86 
Diagonal matrix, 48, 96 
Diagonal! quadratic form, 127 
Diagonalizing matrix, 112 
Difference of two sets, 246 
Difference operator A, 590 
Differentiable scalar field, 258 
Differentiable vector field, 270 
Differential equations, 142 
first-order linear, 143 
homogeneous, 146 
nth-order linear, 145 
partial, 283 
power-series solutions of, 169, 220 
systems of, 191 
Dimension of a linear space, 12 
Dimensionality theorem, 147 
DIRAC, PAUL A. M., 108 (Exercise 8) 
Directional derivative, 254 
Disconnected set, 333 
Discontinuity, 366, 517 
Discrete distribution, 521 
Discrete random variable, 521 


Index 


Discriminant of a quadratic form, 134 
Disk, 244 
Distance in a Euclidean space, 26 
Distribution function: 
binomial, 521 
Cauchy, 530 
continuous, 525 
discrete, 521 
exponential, 533 
joint, 543 
mixed, 539 
normal, 535 
normal bivariate, 555 (Exercise 9) 
of a function of a random variable, 541, 550 
of a one-dimensional random variable, 515 
of a two-dimensional random variable, 543 
Poisson, 522 
standard normal, 537 
uniform, 526 
Divergence of a vector field, 441, 460 
Divergence theorem, 457 
Dot product, 14 
Double integral(s): 
applications of, 374 
evaluation by repeated integration, 358, 367 
existence of, 358, 363 
in polar coordinates, 397 
of a bounded function, 357 
of a step function, 355 
transformation formula for, 394 
Doubly connected region, 391 (Exercise 3) 


Eigenfunction, 99 

Eigenspace, 98 

Eigenvalue, 97 

Eigenvector, 97 

Element of a linear space, 3 

Entries of a matrix, 45, 51 

Equipotential lines and surfaces, 335 

Equivalence of sets, 501 

Error: 
in Lagrange’s interpolation formula, 583 
in linear interpolation, 584 
in Taylor polynomial approximation, 573 
in Taylor’s formula, 258, 270 

Euclidean space, 15 

EULER, LEONARD, 142, 182, 613 

Euler’s constant, 618 (Exercise 2) 

Euler’s gamma function, 184, 413, 620 (Exer- 

cises 7, 9) 
Euler’s summation formula, 613, 615 
Euler’s theorem for homogeneous functions, 
287 
Events, 476 
Exact differential equation, 347 


667 


Existence theorem(s): 
for determinants, 90 
for first-order linear differential equations, 
143 
for first-order linear systems of differential 
equations, 213, 219, 220 
for first-order nonlinear systems of differ- 
ential equations, 229 
for implicit functions, 237 
for integral equations, 239 
for interpolation polynomials, 580 
for linear nth-order differential equations, 
147 
for linear second-order differential equations, 
143 
for potential functions, 339 
for Taylor polynomials, 572 
for vector fields with prescribed curl, 448 
Expectation (expected value): 
of a function of a random variable, 559 
of a random variable, 556 
Exponential distribution, 533 
Exponential matrix, 197 
Exterior: 
of a Jordan curve, 380 
of a set, 245 
Exterior point, 245 
Extreme-value theorem for continuous scalar 
fields, 321 
Extremum, 304 
second-derivative test for, 312 
with constraints, 314 


Factorial polynomials (factorial nth powers), 
2 


FELLER, WILLIAM, 568 
FERMAT, PIERRE DE, 469 
Finite additivity, 470 
Finite sample space, 473 
Finite set, 502 
Finitely additive measure, 472 
Finitely additive set function, 470 
Fixed point of an operator, 233 
Fixed-point theorem for contraction operators, 
235 
applications of, 237 
Flow integral, 330 
Flux density, 432, 461 
Forward difference operator A, 590 
FOURIER, JOSEPH, 29 
Fourier coefficients, 29 
FRECHET, RENE MAURICE, 258 
FROBENIUS, GEORG, 181 
Frobenius’ method, 180 
FUCHS, LAZARUS, 223 


668 


Function space, 5 
Functional equation: 
for the gamma function, 185 
of Cauchy, 528 
Functions of random variables, 541, 550 
Fundamental theorems for line integrals: 
first, 338 
second, 334 
Fundamental vector product of a surface, 420 


GALLE, JOHANN, 571 
Gamma function, 184, 413, 620 (Exercises 
7, 9) 
GAUSS, KARL FRIEDRICH, 61, 78, 457 
Gauss-Jordan process: 
for calculating determinants, 78 
for solving linear equations, 61 
Gauss’ theorem (divergence theorem), 457 
General solution: 
of a linear differential equation, 148, 156 
of a system of linear equations, 60 
GOMBAUD, ANTOINE (Chevalier de Méré), 469 
Gradient, 259 
GRAM, JORGEN PEDERSEN, 22 
Gram-Schmidt process, 22 
GREEN, GEORGE, 379 
Green’s formula, 387 (Exercise 8) 
Green’s theorem: 
for multiply connected regions, 387 
for simply connected regions, 380 


HADAMARD, JACQUES, 69 
Hadamard matrices, 69 (Exercise 10) 
HAMILTON, WILLIAM ROWAN, 202 
Harmonic function, 445 
Heat equation, 292 (Exercise 1) 
HERMITE, CHARLES, 15, 114, 122, 178 
Hermite differential equation, 178 (Exercise 4) 
Hermitian matrix, 122 
Hermitian operator, 115 
Hermitian symmetry, 15, 114 
HERSCHEL, WILLIAM, 571 
HESSE, LUDWIG OTTO, 308 
Hessian matrix, 308 
Homogeneous function, 287 
Homogeneous linear differential equation, 146 
Homogeneous system: 
of differential equations, 200 
of linear equations, 59 
HUYGENS, CHRISTIAN, 469 


Identity matrix, 54 
Identity transformation, 32 


Index 


Implicit differentiation, 294 
Implicit-function theorem, 237 
Independence: 
in a linear space, 9 
of eigenvectors, 100 
of events, 488 
of nonzero orthogonal elements, 18 
of parametrization, 327, 434 
of path, 333 
of random variables, 547 
Independent trials, 495 
Indicial equation, 183 
Inequality: 
Cauchy-Schwarz, 16 
Chebyshev, 563 
triangle, 17 
Wallis’, 616 
Infinite set, 502 
Initial conditions, 143, 147 
Initial-value problem, 193 
Inner product, 14 
Integral: 
double, 357 
line, 324 
n-fold, 406 
of a matrix function, 193 
surface, 430 
triple, 406 
Integral equation, 239 
Integrating factor, 349 
Interior (inner region) of a Jordan curve, 380 
Interior of a set, 244 
Interior point, 244 
Interpolation: 
by Lagrange’s formula, 580 
by Newton’s formula, 589 
Interval in n-space, 405 
Invariance property: 
of Lagrange interpolation coefficients, 586 
(Exercise 6) 
of line integrals under change of parame- 
trization, 327 
of line integrals under deformation of the 
path, 388 
of surface integrals under change of parame- 
trization, 433 
Invariant subspace, 99 
Inverse function, 39 
Inverse matrix, 66 
Inverse transformation, 39, 41 
Invertible function, 40 
Invertible transformation, 40, 41 
Irrotational vector field, 445 
Isometry, 129 
Isomorphism, 53 
Isothermals, 266 


Iterated integrals, 359, 367, 406 
Iterated limits, 251 (Exercise 2) 


JACOBI, CARL GUSTAV JACOB, 271 
Jacobian determinant, 297, 394, 408, 420 
Jacobian matrix, 271 

Joint probability density, 546 

Joint probability distribution, 543 
Jointly distributed random variables, 543 
JORDAN, CAMILLE, 61, 78, 379 

Jordan curve, 379 

Jordan curve theorem, 380 

Jump discontinuity, 518 


Kernel: 
of a linear transformation, 33 
of an integral equation, 239 
Kinetic energy, 329 
KOLMOGOROV, ANDREI NIKOLAEVICH, 470 


LAGRANGE, JOSEPH LOUIS, 142, 315, 580 
Lagrange interpolation coefficient, 209, 581 
Lagrange interpolation formula, 580 
Lagrange multipliers, 315 
LAPLACE, PIERRE SIMON, 142 
Laplace’s equation, 283, 292 
Laplacian, 292, 444 
Law of large numbers, 565, 566 
Least-square approximation, 573 
LEGENDRE, ADRIEN MARIE, 25, 171 
Legendre polynomials, 25, 174 

graphs of, 175 

normalized, 26 

Rodrigues’ formula for, 176 

zeros of, 180 (Exercise 14) 
Legendre’s differential equation, 171 
LEIBNIZ, GOTTFRIED WILHELM, 142 
Level sets (curves, surfaces), 266 
LEVERRIER, JEAN JOSEPH, 571 
LINDEBERG, JARE W., 567 
Lindeberg’s condition, 568 
Line integrals: 

applications of, 330 

first fundamental theorem for, 338 

independence of parametrization, 327 

independence of path, 333 

notations for, 324 

second fundamental theorem for, 334 
Linear combination, 8 
Linear differential equation, 145 
Linear differential operator, 146 
Linear interpolation, 584 
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Linear operator (transformation), 31 
Linear space (vector space), 3 

Linear subspace, 8 

Linear system, 58, 192 

Linear transformation, 31 

LIOUVILLE, JOSEPH, 143, 191, 223 
LIPSCHITZ, RUDOLF, 229 

Lipschitz condition, 229 

Lorentz transformation, 69 (Exercise 6) 
Lower integral, 358 

LYAPUNOV, ALEXSANDR MIKHAILOVICH, 567 


Mapping, 31, 393, 550 
MARKOV, ANDREI ANDREEVICH, 470 
Mass, 330, 373, 414, 431 
Mass interpretation of probability, 511, 
545 
Matrix: 
adjoint, 122 
adjugate, 92 
algebraic operations on, 52, 54 
block-diagonal, 84 
cofactor, 92 
conjugate, 122 
definition of, 46 
diagonal, 48, 96 
exponential, 197 
Hermitian, 122 
minor of, 87 
nonsingular, 65 
orthogonal, 123 
representation, 46 
self-adjoint, 122 
series, 194 
singular, 67 
skew-Hermitian, 122 
skew-symmetric, 124 
symmetric, 124 
trace of, 106 
transpose of, 91 
unitary, 123 
Max norm, 575 
Maximum of a function: 
absolute, 304 
existence of, 321 
relative, 304 
tests for, 311, 312 
Mean: 
of a function, 374 
of a random variable, 535, 556 
Mean-value theorem for scalar fields, 254 
Measurable sets, 510 
Measure: 
finitely additive, 472 
probability, 474, 506 
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Minimum of a function: 
absolute, 304 
existence of, 321 
relative, 304 
tests for, 311, 312 
Minor of a matrix, 87 
MISES, RICHARD VON, 4/70 
MOBIUS, AUGUSTUS FERDINAND, 455 
Mobius band, 455 
Moment of inertia, 331, 374, 414, 431 
Multiple integrals: 
applications of, 373, 414, 431 
definition of, 357, 406 
evaluation of, 358, 406 
existence of, 363 
transformation formulas for, 394, 408 
Multiply connected region, 384 


NAPIER, JOHN, 572 
Negative definite quadratic form, 310 
Negative (clockwise) direction, 389 
Neighborhood, 244 
Neptune, 571 
NEWTON, ISAAC, 142, 335, 571, 589 
Newton’s interpolation formula, 589 
Newtonian potential, 335 
n-fold integral, 405 
Nonlinear differential equation, 228 
Nonorientable surface, 456 
Nonsingular matrix, 65 
Norm: 

in a Euclidean space, 17 

in a linear space, 233 

max, 575 

of a matrix, 195 

square, 575 


Normal approximation to binomial dis- 


tribution, 538 


Normal bivariate distribution, 555 (Exercise 9) 


Normal derivative, 386 
Norma! distribution, 535 
Normal vector to a surface, 267, 423 
Normed linear space, 233, 574 
Null space, 33 
Nullity, 34 
Numerical integration: 
by Chebshev formulas, 611 
by Cotes’ rule, 612 (Exercise 11) 


by Euler’s summation formula, 613, 615 


by Simpson’s rule, 608, 609 
by the trapezoidal rule, 604 


Odd man out, 508 (Exercises 10, 11) 
Odds, 480 


Index 


One-element set (singleton), 475 
One-to-one correspondence, 501 
One-to-one function, 40 
Open ball, 244 
Open set, 244 
Operator: 

contraction, 235 

curl, 441 

derivative, 32 

divergence, 441 

forward difference, 590 

gradient, 259 

identity, 32 

integration, 32 

Laplacian, 292 

linear, 31 
Ordered basis, 13 
Ordinate set, 360, 368 
Orientable surface, 456 
Orientation of Jordan curve, 389 
Orthogonal basis, 19 
Orthogonal complement, 27 
Orthogonal decomposition theorem, 27 
Orthogonal elements, 18 
Orthogonal functions, 18 
Orthogonal matrix, 123 
Orthogonal operator, 138 
Orthogonal projection, 28 
Orthogonal trajectory, 267 
Orthogonalization theorem, 22 
Orthonormal! set, 18 
Outcome, 476 


PAPPUS OF ALEXANDRIA, 376, 426 
Pappus’ theorems, 376, 428 
Parallel-axis theorem, 378 (Exercise 17) 
Parametric surface, 420 

PARSEVAL, MARK-ANTOINE, 20 
Parseval’s formula, 20 

Partial derivative, 254, 255 

Partial differential equation, 283 
Partition of a rectangle, 354 

PASCAL, BLAISE, 469 

Path in n-space, 323 

PAULI, WOLFGANG, 107 (Exercise 4) 
Pauli spin matrices, 107 (Exercise 4) 
PEANO, GUISEPPE, 223, 611 
Petersburg problem, 562 (Exercise 10) 
PICARD, CHARLES EMILE, 223 
Picard’s method, 223, 227 

Piecewise linear interpolation, 603 
Piecewise smooth path, 324 

POISSON, SIMEON DENIS, 522 

Poisson distribution, 522 

Polar coordinates, 274, 397 
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Polar moment of inertia, 375 Repeated integrals, 359, 406 
Polynomials: Repeated trials, 495 
Chebyshev, 596 RICATTI, VINCENZO, 142 
interpolation, 579, 589 Ricatti’s differential equation, 142 
Legendre, 26, 176, 577 RIEMANN, GEORG FRIEDRICH BERNHARD, 539, 
Taylor, 572 619 
Positive definite quadratic form, 310 Riemann zeta function, 619 (Exercise 3) 
Positive (counterclockwise) direction, 390 RODRIGUES, OLINDE, 176 
Potential energy, 336 Rodrigues’ formula, 176 
Potential function, 335 Rotation, 129 
construction of, 338, 342 Roulette, 558 


existence of, 339 
Newtonian, 335 


on convex sets, 344 Saddle point, 304 
Power series solutions: Sample, 476 
of linear differential equations, 169 Sample space, 473 
of homogeneous linear systems, 220 Sampling, 483, 484 
Probability: Saturn, 571 
conditional, 487 Scalar, 4 
definition of, 474, 506, 511 Scalar field, 243 
Probability density function, 525, 546 total derivative of, 258 
Probability distribution, 515, 543 SCHMIDT, ERHARD, 22 
Probability mass function, 520, 545 SCHWARZ, HERMANN AMANDUS, 16 
Probability measure, 474 Second-derivative test for extrema, 311, 312 
Probability space, 474 Self-adjoint matrix, 122 
Projections, 24, 28 Seminorm, 574 
PTOMELY, CLAUDIUS, 571 interpolation, 575 
PUTZER, E. J., 202 Taylor, 574 
Putzer’s method, 205 Sequential counting, principle of, 483 
Pythagorean formula, 10, 20, 27 Series of matrices, 194 


Series solutions: 
of homogeneous linear systems, 220 


Quadratic form, 127, 310 of linear differential equations, 169 
extreme values of, 137 Set function, 470 
negative definite, 310 Similar matrices, 111 
positive definite, 310 Simple closed curve, 379 
reduction to diagonal form, 129 Simple parametric surface, 420 


Simply connected set, 384 
SIMPSON, THOMAS, 608 


Radius of gyration, 557 Simpson’s rule, 608, 609 
Random variable(s), 512 Singleton (one-element set), 475 
continuous, 525 Singular point: 
discrete, 521, 545 of a differential equation, 146, 181 
functions of, 541 of a mapping, 396 
independent, 547 of a surface, 421 
jointly distributed, 543 Skew-Hermitian matrix, 122 
Range of a linear transformation, 32 Skew-Hermitian operator, 115 
Rank, 34 Skew-symmetric matrix, 124 
Rate of change, 253 Skew-symmetric operator, 115 
Rational function, 249 Smooth path, 323 
Real linear space, 4 Smooth surface, 421 
Regular point of a surface, 421 Solenoidal vector field, 450 
Regular singular point, 181 Solid angle, 463 (Exercise 13) 
Relative maximum, 304 Sphere: 
Relative minimum, 304 area of, 427 


Repeated experiments, 492 volume of (in n-space), 411 
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Spherical coordinates, 293, 410, 414 
Standard deviation, 557 
Standardized norma! distribution, 537 

table of values of, 536 
Standardized random variable, 567 
Stationary point, 304 
Step function, 354, 406 

integral of, 355, 406 
STIRLING, JAMES, 572, 593 
Stirling formula for 1 factorial, 616 
Stirling numbers: 

of first kind, 593 (Exercise 4) 

of second kind, 594 (Exercise 5) 
Stochastic, 512 
Stochastic independence, 488 
Stochastic variable, 512 
STOKES, GEORGE GABRIEL, 438 
Stokes’ theorem, 438, 455 
Strong law of large numbers, 566 
Sturm-Liouville operator, 116 
Subspace, 8 
Summation formula of Euler, 613, 615 
Surface: 

area of, 424 

closed, 456 

equipotential, 335 

nonorientable, 456 

of revolution, 428 

orientable, 456 

parametric, 420 

smooth, 421 
Surface integrals, 430 

applications of, 431 

independence of parametrization, 434 

notations for, 434 
Symmetric difference, 473 (Exercise 10) 
Symmetric matrix, 124 
Symmetric operator, 115 
Systems : 

of differential equations, 197 

of linear algebraic equations, 58 


Tangent plane to a surface, 268, 424 
TAYLOR, BROOK, 571 
Taylor’s formula: 

for scalar fields, 258, 308 

for vector fields, 270 
Taylor polynomials, 571 
Torus, 376 
Total derivative: 

of a scalar field, 258 

of a vector field, 270 
Trace of a matrix, 106 
Trajectory, orthogonal, 267 


Index 


Transformations: 

linear, 31 

of double integrals, 394 

of n-fold integrals, 408 
Trapezoidal rule, 604 
Triangle inequality, 17 
Trigonometric polynomial, 29 
Triple integrals: 

applications of, 414 

in cylindrical coordinates, 410 

in spherical coordinates, 411 


Uncountable set, 502 
Uniform continuity, 321 
Uniform distribution: 
over an interval, 526 
over a plane region, 549 (Exercise 9) 
Over a square, 547 
Uniqueness theorem for determinants, 79 
Uniqueness theorems for differential equa- 
tions: 
first-order linear systems, 226 
first-order nonlinear systems, 229 
linear equations of order n, 147 
matrix differential equations, 198 
Unit sphere in n-space, 135 
volume of, 411 
Unitary matrix, 123 
Unitary operator, 138 
Unitary space, 15 
Upper integral, 358 
Uranus, 571 


Variance, 556 
binomial distribution, 557 
Cauchy distribution, 561 (Exercise 7) 
exponential distribution, 561 (Exercise 7) 
normal distribution, 558 
Poisson distribution, 561 (Exercise 7) 
uniform distribution, 557 
Variation of parameters, 157 
VEBLEN, OSWALD, 380 
Vector equation of a surface, 418 
Vector field, 243 
total derivative of, 270 
Vector space, 4 
Volume: 
of an ordinate set, 360, 369 
of an n-dimensional sphere, 411 


WALLIS, JOHN, 616 
Wallis’ inequality, 616 
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Wave equation, 288 YOUNG, W. H., 258 
Weak law of large numbers, 565 
Winding number, 389 


Work: Zeros: 

and energy principle, 329 of Bessel functions, 188 (Exercise 3) 

as a line integral, 328 of Chebyshev polynomials, 597 
WRONSKI, J. M. HOENE, 95 (Exercise 8), 159 of Legendre polynomials, 177 
Wronskian determinant, 161 Zeta function of Riemann, 619 (Exercise 3) 


Wronskian matrix, 95 (Exercise 8), 159 ZUCKERMAN, HERBERT S., 559 


